# relaybus > relaybus is a reliable, storage-agnostic event bus for NestJS and Node.js: a transactional outbox with per-handler retries, leases, dead letters and ordering keys for PostgreSQL and MongoDB, with Kafka and RabbitMQ transports. relaybus stores every event in the same database transaction as your business data (the transactional outbox pattern), then delivers it to each handler and broker with independent retries, leases and dead letters. Packages: @relaybus/core, @relaybus/nestjs, @relaybus/postgres, @relaybus/mongodb, @relaybus/kafka, @relaybus/rabbitmq, @relaybus/testing, @relaybus/eslint-plugin, @relaybus/otel. Requires Node.js 22.12 or newer; NestJS 11 or 12 for the NestJS module. This file contains the complete relaybus documentation. Index: https://berkcansavur.github.io/relaybus/llms.txt --- Source: https://berkcansavur.github.io/relaybus/ # relaybus ## What is relaybus? relaybus is a reliable, storage-agnostic event bus for NestJS and Node.js that writes every event in the same database transaction as your business data (the transactional outbox pattern) and delivers it to each handler and broker with independent retries, leases and dead letters. ```ts await bus.transaction(async (tx) => { await tx.query('INSERT INTO orders ...'); await bus.publish(new OrderPlaced({ orderId }), { key: orderId }); }); ``` Install it with `pnpm add @relaybus/core @relaybus/nestjs @relaybus/postgres pg` (Node.js 22 or newer, NestJS 11 or 12), then follow [Getting started](https://berkcansavur.github.io/relaybus/getting-started). ## Questions this site answers - [How do I publish events reliably in NestJS?](https://berkcansavur.github.io/relaybus/faq#how-do-i-publish-events-reliably-in-nestjs) - [How to implement the transactional outbox pattern in NestJS?](https://berkcansavur.github.io/relaybus/faq#how-to-implement-the-transactional-outbox-pattern-in-nestjs) - [How to avoid losing events between the database and Kafka?](https://berkcansavur.github.io/relaybus/faq#how-to-avoid-losing-events-between-the-database-and-kafka) - [relaybus vs @nestjs/event-emitter, @nestjs/cqrs, @nestjs/outbox, BullMQ and pg-boss](https://berkcansavur.github.io/relaybus/comparison) For AI assistants and agents: [llms.txt](https://berkcansavur.github.io/relaybus/llms.txt) is a concise index and [llms-full.txt](https://berkcansavur.github.io/relaybus/llms-full.txt) contains the full documentation as Markdown. --- Source: https://berkcansavur.github.io/relaybus/getting-started # Getting started: NestJS and PostgreSQL in 5 minutes This guide adds relaybus to a NestJS service that already uses PostgreSQL. You need Node.js 22.12 or newer, NestJS 11 or 12 and PostgreSQL 13 or newer. ## 1. Install ```sh pnpm add @relaybus/core @relaybus/nestjs @relaybus/postgres pg pnpm add -D @types/pg ``` ## 2. Register the module ```ts // app.module.ts import { Module } from '@nestjs/common'; import { Pool } from 'pg'; import { RelayBusModule } from '@relaybus/nestjs'; import { PostgresEventStore } from '@relaybus/postgres'; const pool = new Pool({ connectionString: process.env.DATABASE_URL }); @Module({ imports: [ RelayBusModule.forRoot({ store: new PostgresEventStore({ pool }), source: 'orders', }), ], }) export class AppModule {} ``` The module is global. On application bootstrap it creates the tables (unless you set `setupStore: false`) and starts the dispatcher. Call `app.enableShutdownHooks()` in `main.ts` so running handlers finish on SIGTERM. ## 3. Define an event and a handler ```ts // order.events.ts import { defineEvent } from '@relaybus/core'; export class OrderPlaced extends defineEvent<{ orderId: string }>('order.placed.v1') {} ``` ```ts // shipping.handlers.ts import { Injectable } from '@nestjs/common'; import { RelayBusConsumer } from '@relaybus/nestjs'; import type { EventEnvelope } from '@relaybus/core'; import { OrderPlaced } from './order.events'; @Injectable() export class ShippingHandlers { @RelayBusConsumer(OrderPlaced, { retry: { attempts: 5 } }) async ship(event: EventEnvelope<{ orderId: string }>) { // runs after the order transaction commits, retried until it succeeds } } ``` List `ShippingHandlers` in the `providers` of one of your modules. Handlers are discovered automatically. ## 4. Publish inside your transaction ```ts // order.service.ts import { Injectable } from '@nestjs/common'; import { RelayBus } from '@relaybus/core'; import type { PoolClient } from 'pg'; import { OrderPlaced } from './order.events'; @Injectable() export class OrderService { constructor(private readonly bus: RelayBus) {} async place(orderId: string) { await this.bus.transaction(async (tx) => { await tx.query('INSERT INTO orders (id) VALUES ($1)', [orderId]); await this.bus.publish(new OrderPlaced({ orderId }), { key: orderId }); }); } } ``` If the `INSERT` rolls back, the event never existed. If the transaction commits, `ShippingHandlers.ship` runs at least once, retried with exponential backoff and jitter until it succeeds or the event is dead-lettered. ## 5. See it work Call `place('o-1')`, then look at the tables relaybus created: `relaybus_events` holds the event and `relaybus_deliveries` holds one delivery per handler with its status. Make the handler throw to watch the retries. ## Next steps - [Guarantees](https://berkcansavur.github.io/relaybus/guarantees) states exactly what is promised. - [Concepts](https://berkcansavur.github.io/relaybus/concepts/outbox) explains outbox, deliveries, leases, retries, ordering keys and dead letters. - Already using an ORM? See the [Prisma](https://berkcansavur.github.io/relaybus/recipes/prisma), [TypeORM](https://berkcansavur.github.io/relaybus/recipes/typeorm) and [Mongoose](https://berkcansavur.github.io/relaybus/recipes/mongoose) recipes. - Need to reach other services? See [Kafka between services](https://berkcansavur.github.io/relaybus/recipes/kafka-between-services) and [RabbitMQ](https://berkcansavur.github.io/relaybus/recipes/rabbitmq). --- Source: https://berkcansavur.github.io/relaybus/guarantees # Guarantees | Property | Behaviour | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Atomic publish | Events are written in the caller's transaction. Rolled back with it, committed with it. | | No lost events | Committed events are delivered to every subscribed handler and route, retried until they succeed or are dead-lettered. Nothing is dropped silently. | | Delivery | At-least-once to every handler. A handler subscribed with `transactional: true` that writes through its transaction takes effect exactly once. | | Independent handlers | Each handler has its own delivery record: one failing handler never re-runs the others. | | Ordering | Optional per key: events with the same `key` are handled one at a time per handler, in commit order. No global ordering. | | Concurrency | Deliveries are leased (`SKIP LOCKED` on SQL, compare-and-set on MongoDB). Every state change is fenced by the lease token, so a stalled worker cannot overwrite the worker that took over. | | Retries | Capped exponential backoff with full jitter, per handler. Non-retryable errors and exhausted deliveries go to dead letters you can inspect and requeue. | | Shutdown | Running handlers finish or are released without losing an attempt. | ## What relaybus does not claim - **Exactly-once side effects outside your database.** An HTTP call or an email can repeat when a handler fails after the call and is retried. Send an idempotency key, or deduplicate on `event.id`. - **Global ordering.** Order holds per key and per handler. Across keys, and across brokers, it does not. - **Delivery to systems that are down forever.** Events wait in the outbox, are retried with backoff and end in dead letters when retries are exhausted. You decide when to requeue. ## Where each guarantee comes from - Atomic publish: the event and one delivery per subscribed handler are inserted in your transaction. See [Transactional outbox](https://berkcansavur.github.io/relaybus/concepts/outbox). - No lost events and retries: see [Deliveries](https://berkcansavur.github.io/relaybus/concepts/deliveries), [Retries and backoff](https://berkcansavur.github.io/relaybus/concepts/retries) and [Dead letters](https://berkcansavur.github.io/relaybus/concepts/dead-letters). - Concurrency and fencing: see [Leases and fencing](https://berkcansavur.github.io/relaybus/concepts/leases). - Ordering: see [Ordering keys](https://berkcansavur.github.io/relaybus/concepts/ordering-keys). - Transport guarantees (publisher confirms, offsets committed after the inbox write): see [@relaybus/kafka](https://berkcansavur.github.io/relaybus/packages/kafka) and [@relaybus/rabbitmq](https://berkcansavur.github.io/relaybus/packages/rabbitmq). ## How it is verified Every store runs the same contract suite from `@relaybus/testing` against a real database: commit-order sequencing, concurrent claimers in separate processes, fencing, starvation and data fidelity. Transports are tested against real Kafka and RabbitMQ brokers, including broker restarts and poison messages. The repository enforces 100% statement, branch, function and line coverage and runs mutation testing, CodeQL and OpenSSF Scorecard in CI. --- Source: https://berkcansavur.github.io/relaybus/concepts/outbox # Transactional outbox ## The problem: dual writes Publishing an event after a database write is a dual write. A crash, a broker outage or a rollback between the two writes either loses the event or announces a change that never happened. In-process emitters lose events on restart, and queue libraries on Redis cannot join your SQL transaction. ## The pattern The transactional outbox writes the event to a table in the same database transaction as the business data. A separate relay reads the table after commit and delivers the events. The database transaction is the single source of truth: commit keeps both, rollback discards both. ## How relaybus implements it 1. `publish` writes the event and **one delivery per subscribed handler or route** into the active transaction. Events with a key get a per-key sequence that follows commit order. 2. The dispatcher in every process claims due deliveries with a [lease](https://berkcansavur.github.io/relaybus/concepts/leases), runs the handler, renews the lease while it runs and marks the delivery completed. 3. A failure reschedules the delivery with [backoff](https://berkcansavur.github.io/relaybus/concepts/retries), or [dead-letters](https://berkcansavur.github.io/relaybus/concepts/dead-letters) it after the last attempt. A worker that crashes leaves an expired lease, and another worker takes the delivery over. 4. Routes forward events to Kafka or RabbitMQ after commit, so a broker outage only delays them. Subscriptions store received events by id before acknowledging them, which makes consumption idempotent (the inbox pattern). ## Which transaction does `publish` join? `publish` finds the transaction in this order: the `transaction` option, the transaction opened by `bus.transaction()`, then your `transactionResolver` (for example from `@nestjs-cls/transactional`). With none, it throws `TransactionRequiredError` unless you opt into `transactionMode: 'optional'`, where the event stays durable but is no longer atomic with your writes. ```ts await bus.transaction(async (tx) => { await tx.query('INSERT INTO orders ...'); await bus.publish(new OrderPlaced({ orderId }), { key: orderId }); }); ``` ## Related - [Deliveries](https://berkcansavur.github.io/relaybus/concepts/deliveries) - [Guarantees](https://berkcansavur.github.io/relaybus/guarantees) - [How to implement the transactional outbox pattern in NestJS?](https://berkcansavur.github.io/relaybus/faq#how-to-implement-the-transactional-outbox-pattern-in-nestjs) --- Source: https://berkcansavur.github.io/relaybus/concepts/deliveries # Deliveries A **delivery** is one event for one handler (or one transport route). When you publish an event, relaybus inserts the event row and one delivery row for every handler or route subscribed to that event type. The `deliveries` table has a unique `(event_id, handler)` constraint. ## Status A delivery is `pending` (waiting for its due time), `processing` (leased by a worker), `completed` or `dead`. Each delivery tracks its attempts, the time it becomes available again, the lease owner, token and expiry, and the last error. ## Why per handler Each handler has its own delivery record, so one failing handler never re-runs the others. A slow or broken email handler cannot cause the stock handler to run twice, and each handler can have its own retry policy, timeout and concurrency: ```ts @RelayBusConsumer(OrderPlaced, { retry: { attempts: 5, initialDelay: '2s', maxDelay: '5m' }, timeout: '30s', concurrency: 4, }) async reserveStock(event: EventEnvelope<{ orderId: string }>) {} ``` Register every handler in each process that publishes the event: deliveries are created at publish time. ## Routes are deliveries too A route forwards an event to a transport (Kafka or RabbitMQ). It is retried until the broker accepts it (unlimited attempts by default, 30 second publish timeout), exactly like a handler. ## Inspecting deliveries ```ts const dead = await bus.listDeliveries({ status: 'dead', handler: 'Shipping.ship' }); const stats = await bus.statistics(); // pending, processing, completed, dead, oldestDueAt ``` Completed deliveries are kept for 7 days by default (`retention.completed`), which is also the deduplication horizon for ids. ## Related - [Leases and fencing](https://berkcansavur.github.io/relaybus/concepts/leases) - [Retries and backoff](https://berkcansavur.github.io/relaybus/concepts/retries) - [Dead letters](https://berkcansavur.github.io/relaybus/concepts/dead-letters) --- Source: https://berkcansavur.github.io/relaybus/concepts/leases # Leases and fencing Several processes can run the dispatcher at once. They compete for due deliveries using **leases**. ## Claiming - **PostgreSQL**: one statement chooses candidates with `FOR UPDATE SKIP LOCKED` and leases them (`processing`, `attempts + 1`, a new lease token). - **MongoDB**: compare-and-set updates claim a delivery atomically. A lease lasts 30 seconds by default (`dispatcher.leaseDuration`) and is renewed every third of that while the handler runs. ## Crashes and takeover A worker that crashes leaves an expired lease. Any other worker claims the delivery after it expires. If the handler outlives its lease, the handler's `context.signal` is aborted; the retry waits for the abandoned run to settle, so runs never overlap. ## Fencing `complete`, `reschedule`, `bury`, `release` and `extendLease` apply only when the delivery is `processing` **and** the lease token matches. A worker that lost its lease changes nothing, so a stalled worker cannot overwrite the worker that took over. ## Transactional handlers With `transactional: true` the handler runs in a store transaction that also completes the delivery. Its writes through `context.transaction` and the completion commit together, which gives effectively-once processing for database effects. Such a handler must finish within the lease duration or its `timeout`. ## Shutdown On `stop`, the dispatcher stops claiming and waits for running handlers for four fifths of the stop timeout (`stop({ timeout })`, default `dispatcher.shutdownTimeout`, 30 seconds), then aborts the rest and releases them without consuming an attempt. The whole stop, closing transports and the store included, ends at the timeout: deliveries still in flight then (their release did not answer) are abandoned with their lease, which expires after `leaseDuration`, and another process runs them again. At-least-once holds either way. ## Related - [Deliveries](https://berkcansavur.github.io/relaybus/concepts/deliveries) - [Guarantees](https://berkcansavur.github.io/relaybus/guarantees) --- Source: https://berkcansavur.github.io/relaybus/concepts/retries # Retries and backoff Every handler delivery is retried on failure with **capped exponential backoff and full jitter**. The defaults are 10 attempts, a 1 second initial delay, a factor of 2 and a 10 minute cap. ```ts @RelayBusConsumer(OrderPlaced, { retry: { attempts: 5, initialDelay: '2s', maxDelay: '5m' }, timeout: '30s', }) async ship(event: EventEnvelope<{ orderId: string }>) {} ``` `attempts: Infinity` retries forever. Durations are milliseconds or strings such as `'250ms'`, `'30s'`, `'5m'`, `'1h'`. ## Non-retryable errors Throw `NonRetryableError` to dead-letter a delivery at once. Schema validation failures are never retried. ## Timeouts and cancellation A handler `timeout` aborts `context.signal`. Use the signal to stop work when a timeout, lease loss or shutdown happens. ## Routes Routes to Kafka or RabbitMQ retry without a limit by default (1 second initial delay, factor 2, capped at 5 minutes) so a broker outage only delays events. ## Retries and ordering For keyed events, a retrying head blocks its successors for the same handler and key; a dead-lettered head releases them. See [Ordering keys](https://berkcansavur.github.io/relaybus/concepts/ordering-keys). ## Related - [Dead letters](https://berkcansavur.github.io/relaybus/concepts/dead-letters) - [Testing with FakeClock](https://berkcansavur.github.io/relaybus/recipes/testing-with-fakeclock) --- Source: https://berkcansavur.github.io/relaybus/concepts/ordering-keys # Ordering keys Pass a `key` when publishing to get ordering for related events, for example one order or one account: ```ts await bus.publish(new OrderPlaced({ orderId }), { key: orderId }); ``` Or declare the key once, on the event, so no publish can forget it. The function is typed from the payload: a misspelled field or a value that is not a string fails to compile. ```ts export class OrderPlaced extends defineEvent('order.placed.v1', { schema: orderPlacedSchema, key: (payload) => payload.orderId, }) {} // or, without a schema export class OrderShipped extends defineEvent<{ orderId: string }>('order.shipped.v1', { key: (payload) => payload.orderId, }) {} await bus.publish(new OrderPlaced({ orderId, total })); // key = orderId ``` Precedence: an explicit `key` option wins over the definition's key; in `publishAll`, an explicit `key` applies to every event of the call, otherwise each event takes its own definition's key (or none). The key is computed from the validated payload (the schema's output) and must be a non-empty string, otherwise `publish` throws a `ConfigurationError`. It works the same through `bus.forCatalog(...)` views. ## What is guaranteed Events with the same `key` are handled **one at a time per handler, in commit order**. There is no global ordering across keys. ## How it works - On append, the key's counter row is incremented inside your transaction. The row lock is held until the transaction ends, so two transactions publishing the same key are serialized and the sequence equals the commit order. - A keyed delivery is claimed only when no earlier delivery of the same handler and key is pending or processing. - A retrying or leased head blocks its successors. A dead-lettered head releases them. - Every route to one destination of a transport is one handler, so the events of a key sent to one topic or exchange keep their commit order across event types. ## Costs and tips - Keep transactions that publish keyed events short: the counter row stays locked until commit. - Publish all keys of a transaction in one `publishAll` call. The store locks keys in sorted order, which avoids deadlocks between transactions. - Pick keys with bounded cardinality: one counter row is kept per key. - Across Kafka, the key is the partition key. With RabbitMQ, order is not preserved unless you set `ordered: true`; see [@relaybus/rabbitmq](https://berkcansavur.github.io/relaybus/packages/rabbitmq). ## Related - [Retries and backoff](https://berkcansavur.github.io/relaybus/concepts/retries) - [Guarantees](https://berkcansavur.github.io/relaybus/guarantees) --- Source: https://berkcansavur.github.io/relaybus/concepts/event-catalogs # Compile-time safety with event catalogs An event with no handler is the quietest failure in an event-driven system: the publish succeeds, nothing runs, and nobody notices until a customer asks why the shipping email never arrived. A **catalog** groups the events of one domain so that the compiler, and then the application boot, tell you before production does. Catalogs are **optional, stricter checking** on top of the default style. Method-level `@RelayBusConsumer(Event)` handlers already have their parameter type checked against the event at compile time, need no class-level setup, and can mix events of any domain in one class; start there. Add a catalog when a team wants the compiler and the boot to prove that every event of a shared contract has a handler or a route. Compiler output below is abridged: `OrderEvents` stands for the expanded type of the catalog that TypeScript prints. ## Before: nothing connects events and handlers ```ts class OrderPlaced extends defineEvent<{ orderId: string }>('order.placed.v1') {} class OrderShipped extends defineEvent<{ orderId: string; carrier: string }>('order.shipped.v1') {} @Injectable() class ShippingHandlers { @RelayBusConsumer(OrderPlaced) onOrderPlaced(event: EventEnvelope<{ orderId: string }>) {} // OrderShipped is forgotten. Everything compiles, the app boots, shipments are silently ignored. } ``` ## After: the catalog makes the gaps visible ```ts export const OrderEvents = defineCatalog({ OrderPlaced, OrderShipped }, { name: 'OrderEvents' }); @Injectable() @RelayBusCatalogConsumer(OrderEvents) class ShippingHandlers implements HandlersOf { onOrderPlaced(event: EventOf) {} } ``` ```text error TS2420: Class 'ShippingHandlers' incorrectly implements interface 'HandlersOf'. Property 'onOrderShipped' is missing in type 'ShippingHandlers' but required in type 'HandlersOf'. ``` Add the method and the error is gone. `EventOf` is `EventEnvelope<{ orderId: string; carrier: string }>`, so the payload is checked too: ```ts onOrderPlaced(event: EventEnvelope<{ orderId: number }>) {} ``` ```text error TS2416: Property 'onOrderPlaced' in type 'ShippingHandlers' is not assignable to the same property in base type 'HandlersOf'. Type '(event: EventEnvelope<{ orderId: number; }>) => void' is not assignable to type '(event: EventOf, context: HandlerContext) => unknown'. Types of property 'orderId' are incompatible. Type 'string' is not assignable to type 'number'. ``` ## Handlers as an object Without NestJS, `defineHandlers` declares the handlers of a catalog and `bus.subscribeCatalog` registers them under the stable ids `${prefix}.on${EventName}`: ```ts const handlers = defineHandlers(OrderEvents, { onOrderPlaced: (event) => reserveStock(event.payload.orderId), onOrderShipped: (event) => notify(event.payload.carrier), }); bus.subscribeCatalog(OrderEvents, handlers, { idPrefix: 'fulfilment' }); ``` ```text // onOrderShipped missing error TS2345: Argument of type '{ onOrderPlaced: () => undefined; }' is not assignable to parameter of type 'HandlerShapeOf & MatchingHandlers<...>'. Property 'onOrderShipped' is missing in type '{ onOrderPlaced: () => undefined; }' but required in type 'HandlerShapeOf'. // onOrderRefunded is not an event of the catalog error TS2322: Type '() => undefined' is not assignable to type 'never'. ``` A consumer that deliberately handles only some events says so with `HandlersOf`, `defineHandlers(OrderEvents, handlers, { events: ['OrderShipped'] })` or `bus.subscribeCatalog(OrderEvents, handlers, { events: ['OrderShipped'] })`; see the microservices section below. ## Publishing only what belongs to the catalog ```ts const orders = bus.forCatalog(OrderEvents); await orders.publish(new OrderPlaced({ orderId }), { transaction }); await orders.publish(new UserRegistered({ userId })); // compile error ``` ```text error TS2345: Argument of type 'UserRegistered' is not assignable to parameter of type 'OrderPlaced | OrderShipped'. Types of property 'payload' are incompatible. Type '{ userId: number; }' is missing the following properties from type '{ orderId: string; carrier: string; }': orderId, carrier ``` TypeScript compares classes by shape, so two events with identical payload types look the same to the compiler. The view therefore also checks the event type at runtime and throws a `ConfigurationError` before anything is stored. The existing `bus.publish` is unchanged. ## The boot check Types cannot see a module that was never imported or a handler that was never registered. Pass the catalogs to the bus (`new RelayBus({ catalogs })`, `bus.registerCatalog(catalog)` or `RelayBusModule.forRoot({ catalogs: [OrderEvents] })`) and `start()` verifies that every event has at least one handler or route: ```text ConfigurationError: Catalog events without a handler or route: - OrderEvents.OrderShipped (order.shipped.v1): expected handler method onOrderShipped ``` The check runs before the store is set up or any transport connects, so a failed check has no side effects and the NestJS application simply does not boot. ## Routing a catalog production A service that publishes several events of a catalog to one topic does not need a route per event. `route(produce(catalog, { events }), settings)` turns every selected event into a route with the same `transport`, `destination`, `retry` and `timeout`: ```ts import { produce, route } from '@relaybus/core'; new RelayBus({ store, transports: [kafka], routes: [ route(produce(OrderEvents, { events: ['OrderPlaced', 'OrderCancelled'] }), { transport: 'kafka', destination: 'orders', retry: { initialDelay: '500ms', maxDelay: '30s' }, timeout: '10s', }), { event: OrderShipped, transport: 'kafka', destination: 'shipments' }, // a plain route still works ], }); ``` - **Same as one route per event.** The result is exactly the routes you would have written by hand: one handler `relaybus.transport::` per destination, so events of one key reach the broker in commit order whatever their types, and `bus.handlers()` lists it once per event type. Without `events`, every event of the catalog is routed. - **`destination` is optional.** Without it every event goes to its own event type, as a plain route does. With it, all selected events share it. - **It covers `produce`.** A route made this way is a route like any other, so `catalogs: [produce(OrderEvents, ...)]` and `RelayBusModule.forFeature({ produces })` see the events as routed. Routing fewer events than a `produce` declaration lists still fails the boot check, naming the events left out. - **Mix freely.** Catalog routes and plain routes can share a destination when they resolve to the same `retry` and `timeout`, as plain routes always had to. The boot refuses a configuration that cannot work, before any transport connects. The errors extend `ConfigurationError`, so existing `catch` blocks keep matching, and carry their own [code](https://berkcansavur.github.io/relaybus/concepts/error-codes): | Mistake | Error | Code | | ----------------------------------------------------------------------------- | --------------------------- | --------------------------------- | | An event routed twice to one destination, when a catalog route is involved | `DuplicateRouteEventError` | `RELAYBUS_ROUTE_DUPLICATE_EVENT` | | A wire name for an event the catalog (or the route's selection) does not have | `UnknownCatalogEventError` | `RELAYBUS_CATALOG_UNKNOWN_EVENT` | | One event with two wire names on a transport, or two events with one | `WireTypeCollisionError` | `RELAYBUS_WIRE_TYPE_COLLISION` | | Wire names for a transport or codec that cannot map type names | `WireTypesUnsupportedError` | `RELAYBUS_WIRE_TYPES_UNSUPPORTED` | Two plain routes to the same event and destination keep failing with the `ConfigurationError` they always did. An unknown event name in `produce(catalog, { events })` does not compile and fails with the `ConfigurationError` of `produce`. ## Wire names next to the catalog and the route Some wire formats name event types differently from relaybus (`Order Placed` instead of `order.placed.v1`). Declare those names where the event or the route is, not only on the codec: ```ts export const OrderEvents = defineCatalog( { OrderPlaced, OrderCancelled, OrderShipped }, { name: 'OrderEvents', wireTypes: { OrderPlaced: 'Order Placed', OrderCancelled: 'Order Cancelled' }, }, ); new RelayBus({ store, transports: [ new KafkaTransport({ kafka, codec: jsonEnvelope({ type: { body: 'eventName' }, id: { body: 'messageId' }, payload: { body: 'data' }, }), }), ], routes: [ route(produce(OrderEvents), { transport: 'kafka', destination: 'orders', wireTypes: { OrderShipped: 'Order Shipped' }, // the route adds to the catalog's names, and wins on a clash }), ], }); ``` The names of the events a transport sends reach its codec when the bus is created, and the codec writes and reads those events under them, byte for byte as `jsonEnvelope`'s own `wireTypes` would (a contract test in the Kafka and RabbitMQ packages proves it). Keys are catalog event names, checked by the compiler; `route` can name only the events it routes. Events without a name travel under their event type. Rules: - A codec has one set of names per transport: the same event cannot have two names on a transport, and no two events can share one. An event without a name travels as its event type, so a name equal to the type of another event the transport sends is refused too (`WireTypeCollisionError`). A clash with the `wireTypes` of the codec itself fails too (`WireTypeCollisionError`); naming the same pair in both places is fine. - The transport needs a codec that maps type names: `jsonEnvelope` with a type at `{ body }` or `{ header }`. The default CloudEvents codecs, a `fixed` type and a hand-written codec refuse wire names at boot (`WireTypesUnsupportedError`). A transport written outside relaybus takes them through the optional `declareWireTypes` method of `Transport`, which checks the names and returns the function that applies them, so a refused configuration changes no transport. - The names live in the transport's codec, so they belong to the transport, not to a bus: two buses that use one transport instance cannot declare wire names for it (`WireTypesUnsupportedError`; give each bus its own transport), and a transport that already connected or subscribed refuses them. Buses that share a transport without declaring any are unaffected. - Wire names of a catalog are used only for the events a route sends. A service that only consumes a catalog has no route to declare them, so give its codec `wireTypes: catalogWireTypes(OrderEvents)`, which lists the catalog's names by event type; names that only a route declared go next to it. ## NestJS: module contracts In NestJS the recommended form is a contract per feature module: `RelayBusModule.forFeature({ consumes: [OrderEvents], produces: [produce(BillingEvents)] })` in the module's `imports`. At boot every consumed event needs a method-level `@RelayBusConsumer` handler among the providers of that module (free method names, any number of classes), and every produced event a route; otherwise the application fails to start naming the module and each missing event. See [@relaybus/nestjs](https://berkcansavur.github.io/relaybus/packages/nestjs#module-contracts-relaybusmodule-forfeature). ## Microservices: shared catalogs Between services, a catalog is a contract: it lives in a shared package, one service produces an event and another consumes it. Each service only covers its part, so instead of a bare catalog (every event needs a handler or route here) it declares its role per event. ```ts // @acme/contracts export const OrderEvents = defineCatalog( { OrderPlaced, OrderCancelled, OrderShipped }, { name: 'OrderEvents' }, ); ``` **Producer service** (orders): publishes `OrderPlaced` and `OrderCancelled`, handles `OrderShipped`. ```ts const bus = new RelayBus({ store: new PostgresEventStore({ pool }), transports: [kafka], routes: [ route(produce(OrderEvents, { events: ['OrderPlaced', 'OrderCancelled'] }), { transport: 'kafka', destination: 'orders', }), ], subscriptions: [{ transport: 'kafka', destinations: ['shipments'] }], catalogs: [ produce(OrderEvents, { events: ['OrderPlaced', 'OrderCancelled'] }), consume(OrderEvents, { events: ['OrderShipped'] }), ], }); bus.subscribeCatalog( OrderEvents, { onOrderShipped: (event) => markShipped(event.payload.orderId) }, { events: ['OrderShipped'] }, ); const orders = bus.forCatalog(OrderEvents, { events: ['OrderPlaced', 'OrderCancelled'] }); await orders.publish(new OrderPlaced({ orderId, total }), { transaction, key: orderId }); ``` **Consumer service** (shipping, NestJS): handles `OrderPlaced` and `OrderCancelled`, publishes `OrderShipped`. ```ts @Injectable() @RelayBusCatalogConsumer(OrderEvents, { events: ['OrderPlaced', 'OrderCancelled'], transactional: true, }) export class ShippingHandlers { onOrderPlaced(event: EventOf, context: HandlerContext) {} onOrderCancelled(event: EventOf, context: HandlerContext) {} } RelayBusModule.forRoot({ store: new MongoEventStore({ client }), transports: [kafka], subscriptions: [{ transport: 'kafka', destinations: ['orders'] }], routes: [{ event: OrderShipped, transport: 'kafka', destination: 'shipments' }], catalogs: [ consume(OrderEvents, { events: ['OrderPlaced', 'OrderCancelled'] }), produce(OrderEvents, { events: ['OrderShipped'] }), ], }); ``` What each declaration checks: | Declaration | Each listed event needs | Does not count | | ------------------------------- | ----------------------- | --------------- | | `consume(catalog, { events? })` | a local handler | a route | | `produce(catalog, { events? })` | a route to a transport | a local handler | | `catalog` (bare) | a handler or a route | | The compiler catches the mistakes it can see: ```text // unknown event name error TS2322: Type '"OrderRefunded"' is not assignable to type '"OrderPlaced" | "OrderCancelled" | "OrderShipped"'. // subset missing handler: events lists OrderShipped but the handlers object has no onOrderShipped error TS2345: Argument of type '{ onOrderPlaced: () => undefined; }' is not assignable to parameter of type 'HandlersOf'. // publishing an event the view does not produce error TS2345: Argument of type 'OrderPlaced' is not assignable to parameter of type 'OrderShipped'. // @RelayBusCatalogConsumer(OrderEvents, { events: ['OrderPlaced', 'OrderShipped'] }) on a class without onOrderShipped error TS1238: Unable to resolve signature of class decorator when called as an expression. ``` and the boot check catches the rest, per event: ```text ConfigurationError: Catalog events this service does not cover: - OrderEvents.OrderPlaced (order.placed.v1): consumed here, needs handler method onOrderPlaced - OrderEvents.OrderShipped (order.shipped.v1): produced here, needs a route to a transport; a local handler does not deliver it to other services ``` On the consuming side, events arrive through the transport (`bus.receive`). They are matched to the catalog handlers by event type, stored once per event id (a broker redelivery is a no-op) and validated against the event's schema before the handler runs; a payload that fails validation is dead-lettered. Every route to one destination shares one queue per key, so events of one key reach the broker in publish order whatever their types, even when a send is retried; the consumer receives them in that order when the broker keeps it (Kafka: same partition, which the key decides; RabbitMQ: `ordered: true`). On the consuming side, order holds per handler: the handlers of two event types run independently and can interleave; put a version or timestamp in the payload when one handler depends on another's effect. ## Related - [Deliveries](https://berkcansavur.github.io/relaybus/concepts/deliveries) - [@relaybus/core](https://berkcansavur.github.io/relaybus/packages/core) - [@relaybus/nestjs](https://berkcansavur.github.io/relaybus/packages/nestjs) --- Source: https://berkcansavur.github.io/relaybus/concepts/dead-letters # Dead letters A delivery becomes **dead** when it exhausts its attempts or its handler throws `NonRetryableError`. Dead letters are kept until you requeue or purge them (`retention.dead` is off by default). ## Inspect and requeue ```ts const dead = await bus.listDeliveries({ status: 'dead', handler: 'Shipping.ship' }); await bus.requeue(dead.map((delivery) => delivery.id)); ``` Each delivery keeps its last error as JSON. `bus.statistics()` reports the number of pending, processing, completed and dead deliveries and the oldest due time. In NestJS, `RelayBusHealth.check()` also reports `lagMilliseconds`, a good alert signal. ## Dead letters and ordering For a keyed event, a dead-lettered head releases the events behind it. A requeued head runs again; if a later event already ran, the head runs after it. ## Broker dead letters Transports have their own dead-letter handling for messages that can never succeed: - Kafka copies malformed records to `.dead` by default. - RabbitMQ dead-letters poison messages to `.dead`. See [@relaybus/kafka](https://berkcansavur.github.io/relaybus/packages/kafka) and [@relaybus/rabbitmq](https://berkcansavur.github.io/relaybus/packages/rabbitmq). ## Related - [Retries and backoff](https://berkcansavur.github.io/relaybus/concepts/retries) - [Deliveries](https://berkcansavur.github.io/relaybus/concepts/deliveries) --- Source: https://berkcansavur.github.io/relaybus/concepts/observability # Observability relaybus reports what it does through observer hooks and a tracing hook in `@relaybus/core`. Core depends on no telemetry library; `@relaybus/otel` maps both to OpenTelemetry. ```sh pnpm add @relaybus/otel @opentelemetry/api ``` ```ts import { relayBusOpenTelemetry } from '@relaybus/otel'; const otel = relayBusOpenTelemetry(); // global meter and tracer providers const bus = new RelayBus({ store, observers: [otel.observer], // metrics tracer: otel.tracer, // spans, traceparent/tracestate in event headers backlogInterval: '30s', // pending and oldest-age gauges }); ``` `RelayBusModule.forRoot` takes the same three options. Register the OpenTelemetry SDK (for example `@opentelemetry/sdk-node`) before the bus is created, or pass `meterProvider` and `tracerProvider`. ## Traces One trace runs from the request that publishes an event to the handler in the service that consumes it, through the outbox and the broker: ``` POST /orders (orders service, your span) └─ publish order.placed.v1 producer stored with the event as traceparent └─ send orders.placed producer linked to the publish span; its context goes on the wire └─ process order.placed.v1 consumer billing service, around the handler └─ publish invoice.created.v1 what the handler publishes joins the trace ``` | Span | Kind | Parent | | ---------------------- | -------- | --------------------------------------------------------------- | | `publish ` | producer | the caller's active span | | `send ` | producer | the publish span, from the stored event's headers (also a link) | | `process ` | consumer | the context the event arrived with: the send span, else publish | - The publish span's context is written into the event's `traceparent` and `tracestate` headers inside the publishing transaction, so it survives the outbox, retries and restarts. Headers you pass to `publish` yourself win. - A route sends the event with the send span's context instead; the stored event is not changed. - A local handler runs inside its `process` span, so database calls, HTTP calls and events it publishes become its children. A retry is a new `process` span in the same trace (`relaybus.delivery.attempt`). - Without `@relaybus/otel`, `traceContextOf(event.headers)` from `@relaybus/core` returns the W3C context an event carries, for any other tracer. Span attributes: `messaging.system` (`relaybus` for the outbox, the transport's system such as `kafka` or `rabbitmq` for sends), `messaging.destination.name`, `messaging.operation.name`, `messaging.operation.type`, `messaging.message.id`, `messaging.message.conversation_id` (the correlation id), `messaging.consumer.group.name` (the handler id), `messaging.batch.message_count`, `relaybus.event.type`, `relaybus.delivery.attempt`, and on failure `error.type` with the exception recorded and the status set to error. The default propagator is W3C Trace Context, built in. Pass `propagator: propagation` (from `@opentelemetry/api`) to use the one the SDK configured, for example to add baggage. ## Metrics Names follow the [OpenTelemetry semantic conventions for messaging](https://opentelemetry.io/docs/specs/semconv/messaging/messaging-metrics/) where one exists. Durations are in seconds. `error.type` is the relaybus error code (`RELAYBUS_LEASE_LOST`, see [Error codes](https://berkcansavur.github.io/relaybus/concepts/error-codes)) or the error's class name, and is present only on failures. | Metric | Type | Unit | Recorded | Attributes | | ------------------------------------- | --------- | ------------ | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | `messaging.client.operation.duration` | histogram | `s` | each publish, claim, send and receive | `messaging.system`, `messaging.operation.name`, `messaging.operation.type`, `messaging.destination.name`, `error.type` | | `messaging.client.sent.messages` | counter | `{message}` | each send attempt of a route | `messaging.system`, `messaging.destination.name`, `messaging.operation.name`, `messaging.operation.type`, `error.type` | | `messaging.client.consumed.messages` | counter | `{message}` | each event received from a broker | as above, plus `relaybus.receive.outcome` | | `messaging.process.duration` | histogram | `s` | each local handler run | `messaging.system`, `messaging.destination.name`, `messaging.consumer.group.name`, `messaging.operation.*`, `error.type` | | `relaybus.events.published` | counter | `{event}` | each event passed to `publish` | `messaging.system`, `messaging.destination.name`, `error.type` | | `relaybus.claim.deliveries` | histogram | `{delivery}` | deliveries per claim query | `messaging.system`, `messaging.operation.*` | | `relaybus.claim.lanes` | histogram | `{lane}` | distinct leases (lanes) per claim query | `messaging.system`, `messaging.operation.*` | | `relaybus.end_to_end.latency` | histogram | `s` | from `occurredAt` to a handler or route start | `messaging.system`, `messaging.destination.name`, `messaging.consumer.group.name`, `relaybus.delivery.kind` | | `relaybus.deliveries.retried` | counter | `{delivery}` | each failed attempt rescheduled | `messaging.system`, `messaging.destination.name`, `messaging.consumer.group.name`, `error.type` | | `relaybus.deliveries.dead_lettered` | counter | `{delivery}` | each delivery moved to dead | as above | | `relaybus.lease.lost` | counter | `{delivery}` | each lease lost while a delivery ran | `messaging.system`, `messaging.destination.name`, `messaging.consumer.group.name` | | `relaybus.deliveries.pending` | gauge | `{delivery}` | last backlog reading (`backlogInterval`) | none | | `relaybus.deliveries.oldest_age` | gauge | `s` | last backlog reading (`backlogInterval`) | none | | `relaybus.failures` | counter | `{failure}` | each error relaybus caught and reported | `relaybus.failure.stage`, `error.type` | | Attribute | Values | | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | `messaging.system` | `relaybus` for publish, claim and handlers; the transport's `system` (`kafka`, `rabbitmq`) or name | | `messaging.destination.name` | the event type; for sends and receives the topic, routing key or destination | | `messaging.operation.name` | `publish`, `claim`, `send`, `receive`, `process` | | `messaging.operation.type` | `create`, `receive`, `send`, `process` | | `messaging.consumer.group.name` | the handler id (a route's id is `relaybus.transport::`) | | `relaybus.delivery.kind` | `handler` or `route` | | `relaybus.receive.outcome` | `accepted`, `duplicate`, `ignored`, `invalid`, `error` | | `relaybus.failure.stage` | `claim`, `dispatch`, `complete`, `reschedule`, `bury`, `release`, `extend-lease`, `purge`, `watch`, `backlog`, `observer`, ... | | `error.type` | a relaybus error code, else the error class name | The backlog gauges need `backlogInterval`: each reading is one `statistics()` query, so it is off by default. Read it in one process per database if many share it, or accept the duplicate series. The metrics come from observer hooks you can also implement yourself: `onPublishFinished`, `onClaimFinished`, `onHandlerStarted`, `onHandlerFinished`, `onTransportSent`, `onReceiveFinished`, `onInvalidMessage`, `onBacklog`, next to the lifecycle hooks (`onDeliveryRetryScheduled`, `onDeliveryDead`, `onLeaseLost`, `onFailure`, ...). A hook that no observer implements costs nothing, not even a clock read. ## Prometheus and Grafana through the OpenTelemetry Collector Export OTLP from the service and let the collector expose Prometheus metrics and forward traces: ```ts // telemetry.ts, loaded before the application import { NodeSDK } from '@opentelemetry/sdk-node'; import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http'; import { OTLPMetricExporter } from '@opentelemetry/exporter-metrics-otlp-http'; import { PeriodicExportingMetricReader } from '@opentelemetry/sdk-metrics'; new NodeSDK({ serviceName: 'orders', traceExporter: new OTLPTraceExporter(), metricReader: new PeriodicExportingMetricReader({ exporter: new OTLPMetricExporter() }), }).start(); ``` ```yaml # otel-collector.yaml receivers: otlp: protocols: http: exporters: prometheus: endpoint: 0.0.0.0:8889 otlp/tempo: endpoint: tempo:4317 tls: insecure: true service: pipelines: metrics: receivers: [otlp] exporters: [prometheus] traces: receivers: [otlp] exporters: [otlp/tempo] ``` Prometheus scrapes `otel-collector:8889`. The collector's Prometheus exporter turns dots into underscores and adds unit and counter suffixes, so the series are named `relaybus_deliveries_oldest_age_seconds`, `relaybus_deliveries_dead_lettered_total`, `messaging_process_duration_seconds_bucket` and so on, with `service_name` from the resource. Useful Grafana panels: ```promql # Handler p95 by handler histogram_quantile(0.95, sum by (le, messaging_consumer_group_name) (rate(messaging_process_duration_seconds_bucket[5m]))) # End-to-end latency p99 across services histogram_quantile(0.99, sum by (le, messaging_destination_name) (rate(relaybus_end_to_end_latency_seconds_bucket[5m]))) # Backlog max by (service_name) (relaybus_deliveries_pending) # Send failures by broker destination sum by (messaging_destination_name, error_type) (rate(messaging_client_sent_messages_total{error_type!=""}[5m])) ``` ## Alerts Start with these two; they catch a stuck dispatcher, a dead broker and a failing handler. ```yaml groups: - name: relaybus rules: - alert: RelayBusBacklogStale # The oldest due delivery has waited 5 minutes: nothing is draining the outbox # (dispatcher down, broker unreachable for a route, a key blocked by a retrying event). expr: max by (service_name) (relaybus_deliveries_oldest_age_seconds) > 300 for: 5m labels: { severity: page } - alert: RelayBusDeadLetters # Deliveries are giving up. Inspect with bus.listDeliveries({ status: 'dead' }) and requeue. expr: sum by (service_name, messaging_consumer_group_name, error_type) (increase(relaybus_deliveries_dead_lettered_total[15m])) > 0 labels: { severity: ticket } ``` Further candidates, tuned to your traffic: a rising `rate(relaybus_deliveries_retried_total[5m])` (a dependency is failing before deliveries die), `relaybus_lease_lost_total` above zero (handlers outliving `dispatcher.leaseDuration`, or clock skew), `relaybus_failures_total` by `relaybus_failure_stage` (store errors), and `relaybus_end_to_end_latency_seconds` p99 against your latency objective. --- Source: https://berkcansavur.github.io/relaybus/concepts/error-codes # Error codes Every error class of relaybus has a readonly `code` on its instances, next to its `name`. The code is stable: once released it is never renamed or reused, so alerts, dashboards and retry rules can branch on it. Messages may change between versions; codes do not. ```ts import { RelayBusError } from '@relaybus/core'; try { await bus.publish(new OrderPlaced({ orderId })); } catch (error) { if (error instanceof RelayBusError && error.code === 'RELAYBUS_TRANSACTION_REQUIRED') { // publish inside a transaction } } ``` ## Where codes appear - **Dead letters and retries.** The `lastError` of a delivery (`bus.listDeliveries()`) has a `code` when the failure was a relaybus error, for example a transport that could not publish. - **Observer events.** The `error` of `onDeliveryRetryScheduled`, `onDeliveryDead` and `onFailure` carries the same `code`. - **Broker dead letters.** Kafka adds `relaybus-error-name` and `relaybus-error-code` to the records it copies to `.dead`; RabbitMQ adds `x-relaybus-dead-letter-error-name` and `x-relaybus-dead-letter-error-code` to the messages it moves to `.dead`. Both keep their reason header, and their log lines carry `errorName=` and `code=`. The code header is left out for errors that have no code. - **`toErrorInfo(error)`** adds `code` for relaybus errors only. Errors of your handlers, drivers and brokers keep `name`, `message` and `stack` without a `code`, exactly as before. The field is additive: records written before 0.3.0 have no `code`, so read it as optional. ## Code scheme `RELAYBUS_` + the package for errors outside core (`KAFKA_`, `POSTGRES_`, `RABBITMQ_`) + the error in upper snake case. An error class you derive from `RelayBusError` yourself inherits `RELAYBUS_ERROR` unless it sets its own code. ## Codes | Code | Error class | Package | Meaning | | ------------------------------------------ | ----------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `RELAYBUS_ERROR` | `RelayBusError` | `@relaybus/core` | Base class; any relaybus error without a more specific code. | | `RELAYBUS_CONFIGURATION` | `ConfigurationError` | `@relaybus/core` | Invalid options passed to a relaybus component. | | `RELAYBUS_CATALOG_UNKNOWN_EVENT` | `UnknownCatalogEventError` | `@relaybus/core` | A catalog or catalog route gives a wire name to an event the catalog (or the route's selection) does not have. Extends `ConfigurationError`. | | `RELAYBUS_ROUTE_DUPLICATE_EVENT` | `DuplicateRouteEventError` | `@relaybus/core` | A catalog route sends an event to a destination another route already sends it to. Extends `ConfigurationError`. | | `RELAYBUS_WIRE_TYPE_COLLISION` | `WireTypeCollisionError` | `@relaybus/core` | One event type has two wire names on a transport, or two event types share one (catalogs, routes, codec `wireTypes`). Extends `ConfigurationError`. | | `RELAYBUS_WIRE_TYPES_UNSUPPORTED` | `WireTypesUnsupportedError` | `@relaybus/core` | Wire names were declared for a transport or codec that cannot map type names (use `jsonEnvelope`). Extends `ConfigurationError`. | | `RELAYBUS_TRANSACTION_REQUIRED` | `TransactionRequiredError` | `@relaybus/core` | `publish` ran outside a transaction with `transactionMode: 'required'`. | | `RELAYBUS_TRANSACTION_TIMEOUT` | `TransactionTimeoutError` | `@relaybus/core` | A `bus.transaction` or `@RelayBusTransaction()` method ran past `transactionTimeout` and was rolled back; usually a slow external call inside it. | | `RELAYBUS_TRANSACTION_ISOLATION_CONFLICT` | `TransactionIsolationConflictError` | `@relaybus/core` | A `@RelayBusTransaction({ isolationLevel })` method ran inside a transaction that relaybus did not open at that level; the level of a running transaction cannot change. | | `RELAYBUS_NON_RETRYABLE` | `NonRetryableError` | `@relaybus/core` | A handler asked to dead-letter the delivery at once. | | `RELAYBUS_EVENT_VALIDATION` | `EventValidationError` | `@relaybus/core` | A payload failed its event schema. Never retried. | | `RELAYBUS_INVALID_ENVELOPE` | `InvalidEnvelopeError` | `@relaybus/core` | A message received from a broker is not a valid event. Never retried. | | `RELAYBUS_LEASE_LOST` | `LeaseLostError` | `@relaybus/core` | The worker no longer owns the delivery it was processing. | | `RELAYBUS_HANDLER_TIMEOUT` | `HandlerTimeoutError` | `@relaybus/core` | A handler did not settle within its timeout. Retried. | | `RELAYBUS_DISPATCHER_STOPPED` | `DispatcherStoppedError` | `@relaybus/core` | The bus stopped before the handler finished; the delivery is released. | | `RELAYBUS_ATTEMPTS_EXHAUSTED` | `AttemptsExhaustedError` | `@relaybus/core` | A delivery came back after its lease expired on its last attempt; dead-lettered without running. | | `RELAYBUS_LIFECYCLE` | `LifecycleError` | `@relaybus/core` | A lifecycle method was called in a state that does not allow it, such as starting a bus twice. | | `RELAYBUS_DRAIN_INCOMPLETE` | `DrainIncompleteError` | `@relaybus/core` | `drain` still found due deliveries after its maximum number of batches. | | `RELAYBUS_KAFKA_INVALID_MESSAGE` | `InvalidKafkaMessageError` | `@relaybus/kafka` | A Kafka record is not a valid event. Never retried. | | `RELAYBUS_KAFKA_UNENCODABLE_EVENT` | `UnencodableEventError` | `@relaybus/kafka` | An event cannot be written to Kafka. Never retried. | | `RELAYBUS_KAFKA_PUBLISH_TIMEOUT` | `PublishTimeoutError` | `@relaybus/kafka` | No broker ack within `publishTimeout`. The outbox retries. | | `RELAYBUS_KAFKA_HANDLER_FAILED` | `HandlerFailedError` | `@relaybus/kafka` | A handler failure handed back to the Kafka client. | | `RELAYBUS_KAFKA_TRANSPORT_CLOSED` | `TransportClosedError` | `@relaybus/kafka` | The transport was closed; it no longer publishes or subscribes. | | `RELAYBUS_KAFKA_GROUP_JOIN_TIMEOUT` | `GroupJoinTimeoutError` | `@relaybus/kafka` | The consumer did not join its group within `startupTimeout`. | | `RELAYBUS_POSTGRES_RECONCILE_FAILED` | `ReconcileFailedError` | `@relaybus/postgres` | A claim was refused while giving back an earlier failed claim still fails; the next claim retries. | | `RELAYBUS_RABBITMQ_UNAVAILABLE` | `RabbitMqUnavailableError` | `@relaybus/rabbitmq` | The broker cannot take or confirm a message right now. Retried. | | `RELAYBUS_RABBITMQ_PUBLISH_REJECTED` | `RabbitMqPublishRejectedError` | `@relaybus/rabbitmq` | The broker nacked a published message. Retried. | | `RELAYBUS_RABBITMQ_UNROUTABLE_MESSAGE` | `UnroutableMessageError` | `@relaybus/rabbitmq` | No queue is bound to the routing key. Retried. | | `RELAYBUS_RABBITMQ_MALFORMED_MESSAGE` | `MalformedMessageError` | `@relaybus/rabbitmq` | A received message is not a valid relaybus CloudEvent. Dead-lettered. | | `RELAYBUS_RABBITMQ_UNSERIALIZABLE_PAYLOAD` | `UnserializablePayloadError` | `@relaybus/rabbitmq` | A payload cannot be represented as JSON. Never retried. | | `RELAYBUS_TESTING_CODEC_CONTRACT` | `CodecContractError` | `@relaybus/testing` | A codec contract case does not match: `mismatches` lists one line per difference. Thrown in tests only. | | `RELAYBUS_RABBITMQ_UNENCODABLE_EVENT` | `RabbitMqUnencodableEventError` | `@relaybus/rabbitmq` | The codec cannot write the event (a reserved header name, a type the route does not carry). Never retried. | | `RELAYBUS_RABBITMQ_AMQP_LIMIT_EXCEEDED` | `AmqpLimitExceededError` | `@relaybus/rabbitmq` | A value exceeds an AMQP 0-9-1 protocol limit. Never retried. | A test keeps this table in step with the code: adding, renaming or removing a code fails it until the change is deliberate. --- Source: https://berkcansavur.github.io/relaybus/packages/ # Packages relaybus is published as nine npm packages under the `@relaybus` scope, released together with one shared version. | Package | Purpose | | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | | [`@relaybus/core`](https://berkcansavur.github.io/relaybus/packages/core) | The bus, dispatcher, retry policies and the in-memory store. | | [`@relaybus/nestjs`](https://berkcansavur.github.io/relaybus/packages/nestjs) | `RelayBusModule`, the `@RelayBusConsumer` decorator, lifecycle and health. | | [`@relaybus/postgres`](https://berkcansavur.github.io/relaybus/packages/postgres) | PostgreSQL 13+ store for `pg`, Kysely, Knex, TypeORM, Drizzle and Prisma transactions. | | [`@relaybus/mongodb`](https://berkcansavur.github.io/relaybus/packages/mongodb) | MongoDB 6+ store on multi-document transactions; works with Mongoose sessions. | | [`@relaybus/kafka`](https://berkcansavur.github.io/relaybus/packages/kafka) | Kafka transport for `@confluentinc/kafka-javascript` (recommended) and kafkajs, CloudEvents binary mode. | | [`@relaybus/rabbitmq`](https://berkcansavur.github.io/relaybus/packages/rabbitmq) | RabbitMQ transport with publisher confirms, reconnects, quorum queues and dead letters. | | [`@relaybus/testing`](https://berkcansavur.github.io/relaybus/packages/testing) | The store contract suite, a fake clock and an in-memory transport. | | [`@relaybus/eslint-plugin`](https://berkcansavur.github.io/relaybus/packages/eslint-plugin) | ESLint rules: no calls to other services inside a transaction, no publish outside one. | | [`@relaybus/otel`](https://berkcansavur.github.io/relaybus/packages/otel) | OpenTelemetry metrics, backlog gauges and one trace across services. | ## Typical installs ```sh # NestJS + PostgreSQL pnpm add @relaybus/core @relaybus/nestjs @relaybus/postgres pg # NestJS + MongoDB pnpm add @relaybus/core @relaybus/nestjs @relaybus/mongodb mongodb # add a broker pnpm add @relaybus/kafka kafkajs pnpm add @relaybus/rabbitmq amqplib # tests and lint rules pnpm add -D @relaybus/testing vitest pnpm add -D @relaybus/eslint-plugin # metrics and traces pnpm add @relaybus/otel @opentelemetry/api ``` Bring your own store: implement the `EventStore` interface and run the contract suite from `@relaybus/testing` against it. --- Source: https://berkcansavur.github.io/relaybus/packages/core # @relaybus/core @relaybus/core is the engine of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: it writes events in the same database transaction as your data (the transactional outbox pattern) and delivers them to every handler with per-handler retries, leases, dead letters and ordering keys. The relaybus engine: transactional publishing, per-handler deliveries, leased dispatching, retries with jitter, dead letters, ordering keys and transport routing. Most applications install it together with a store and, for NestJS, the module: ```sh pnpm add @relaybus/core @relaybus/nestjs @relaybus/postgres pg ``` | Package | Use it for | | ------------------------------------------------------------------------ | ------------------------------------------------------------- | | [`@relaybus/nestjs`](https://www.npmjs.com/package/@relaybus/nestjs) | `RelayBusModule` and the `@RelayBusConsumer` decorator | | [`@relaybus/postgres`](https://www.npmjs.com/package/@relaybus/postgres) | PostgreSQL store (pg, Prisma, TypeORM, Knex, Kysely, Drizzle) | | [`@relaybus/mongodb`](https://www.npmjs.com/package/@relaybus/mongodb) | MongoDB store (driver or Mongoose sessions) | | [`@relaybus/kafka`](https://www.npmjs.com/package/@relaybus/kafka) | Kafka transport between services | | [`@relaybus/rabbitmq`](https://www.npmjs.com/package/@relaybus/rabbitmq) | RabbitMQ transport between services | | [`@relaybus/testing`](https://www.npmjs.com/package/@relaybus/testing) | Contract suite, fake clock, in-memory transport | ## With NestJS ```ts // order.events.ts export class OrderPlaced extends defineEvent<{ orderId: string }>('order.placed.v1') {} // app.module.ts @Module({ imports: [RelayBusModule.forRoot({ store: new PostgresEventStore({ pool }) })] }) export class AppModule {} // shipping.handlers.ts: consume with a decorator @Injectable() export class ShippingHandlers { @RelayBusConsumer(OrderPlaced, { retry: { attempts: 5 } }) async ship(event: EventEnvelope<{ orderId: string }>) { /* runs after the order transaction commits */ } } // order.service.ts: produce inside your transaction await this.bus.transaction(async () => { await this.orders.insert(order); await this.bus.publish(new OrderPlaced({ orderId: order.id }), { key: order.id }); }); ``` See the [project README](https://github.com/berkcansavur/relaybus#readme) for guarantees and architecture. ## Without a framework ```ts import { InMemoryEventStore, RelayBus, defineEvent } from '@relaybus/core'; class OrderPlaced extends defineEvent<{ orderId: string }>('order.placed.v1') {} const bus = new RelayBus({ store: new InMemoryEventStore(), source: 'orders' }); bus.subscribe( OrderPlaced, async (event, context) => { console.log(event.payload.orderId, context.attempt); }, { id: 'orders.log' }, ); await bus.start(); await bus.transaction(async () => { await bus.publish(new OrderPlaced({ orderId: 'o-1' }), { key: 'o-1' }); }); ``` `bus.transaction(work, { isolationLevel: 'serializable', maxAttempts })` runs the transaction at an isolation level (PostgreSQL stores) and reruns `work` after a serialization failure; `work` may then run more than once. See [Serializable transactions](https://berkcansavur.github.io/relaybus/recipes/serializable-transactions). Use a database store in production: [`@relaybus/postgres`](https://berkcansavur.github.io/relaybus/packages/postgres) or [`@relaybus/mongodb`](https://berkcansavur.github.io/relaybus/packages/mongodb). `InMemoryEventStore` refuses to start when `NODE_ENV=production` unless `allowInProduction: true`. ## `new RelayBus(options)` | Option | Default | Description | | ----------------------------------------- | --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | `store` | required | `EventStore` implementation. | | `source` | `'relaybus'` | Service name recorded on every event. | | `transactionMode` | `'required'` | `'optional'` lets `publish` open its own transaction. | | `transactionResolver` | — | Returns the caller's active transaction, e.g. from `@nestjs-cls/transactional`. | | `transactionTimeout` | `'5s'` | Budget of every `bus.transaction`; the database enforces it where it can, an overrun rolls back with `TransactionTimeoutError`. | | `unroutedEvents` | `'warn'` | `'throw'` rejects events without any handler or route. | | `retry` | 10 attempts, `1s`, ×2, max `10m`, full jitter | Default handler retry policy. `attempts: Infinity` retries forever. | | `handlerTimeout` | `false` | Default handler timeout. | | `dispatcher.enabled` | `true` | Run handlers in this process. | | `dispatcher.concurrency` | `10` | Handlers running at once. | | `dispatcher.batchSize` | concurrency | Deliveries claimed per query. | | `dispatcher.pollInterval` | `'1s'` | Idle wait between claims; retries are picked up on this cadence. | | `dispatcher.leaseDuration` | `'30s'` | Lease length, renewed every third while a handler runs. | | `dispatcher.shutdownTimeout` | `'30s'` | Longest `stop` takes in all (default of `stop({ timeout })`); handlers are aborted and released at four fifths of it. | | `dispatcher.completionBatchWindow` | `'2ms'` | Deliveries that finish within this window are completed with one store call; `0` completes one by one. | | `dispatcher.completionBatchSize` | `200` | Most deliveries completed in one store call. | | `retention.completed` | `'7d'` | Delete events whose deliveries all completed after this; also the deduplication horizon. A completed delivery is deleted at once (see below). | | `retention.dead` | `false` | Delete dead letters after this. | | `retention.interval` | `'1h'` | Cleanup cadence. | | `transports` / `routes` / `subscriptions` | — | Broker integration, see below. | | `setupStore` | `true` | Create tables and indexes on `start`. | | `observers` | `[]` | Lifecycle hooks for logs, metrics and tracing (see Observability below). | | `tracer` | none | Spans around publish, send and handler runs; writes `traceparent`/`tracestate` into published events. `@relaybus/otel` provides one. | | `backlogInterval` | off | Reads `statistics()` this often while running and reports it to `onBacklog`. | | `logger` | `console` | `{ log, warn, error }`. | | `clock`, `generateIdentifier`, `random` | system | Replace in tests. A clock with `schedule` also runs the bus's timers. | Durations are milliseconds or strings such as `'250ms'`, `'30s'`, `'5m'`, `'1h'`, `'7d'`. **How the dispatcher schedules claims.** A claim that fills its batch is followed by the next one at once. Otherwise the dispatcher sleeps until it is woken: by a publish, by the store reporting new deliveries from any process (the PostgreSQL store through `LISTEN/NOTIFY`, the MongoDB store through a change stream), or by settled deliveries once 70% of a batch (of `min(concurrency, batchSize)`) is free again, so slots are refilled in groups rather than one by one. A smaller number of free slots is refilled after 10 ms without a claim, and the poll interval stays the safety net. Wake-ups within 5 ms of a claim are merged into one claim. Settled unkeyed deliveries do not trigger a claim when their handler's last claim got less than it asked for (nothing more was due); a settled keyed delivery always may, since it unblocks the next of its key. ## Publishing `publish(event, options)` and `publishAll(events, options)` write into, in order: `options.transaction`, the transaction of `bus.transaction()`, then `transactionResolver`. Options: `key`, `delay`, `headers`, `correlationId`, and `id` on `publish` for idempotent publishing. Register every handler in each process that publishes the event: deliveries are created at publish time. Declare the ordering key on the event to apply it to every publish: `defineEvent('order.placed.v1', { schema, key: (payload) => payload.orderId })` (or `defineEvent(type, { key })`). The function is typed from the payload, so a wrong field fails to compile. An explicit `key` option overrides it; in `publishAll` an explicit key applies to every event of the call. ## Handlers `subscribe(event, handler, { id, retry, timeout, concurrency, transactional })`. The handler receives the envelope and a context with `attempt`, `maxAttempts`, `deliveryId`, `signal` and, for transactional handlers, `transaction`. - Throw `NonRetryableError` to dead-letter at once. Schema failures are never retried. - `transactional: true` runs the handler in a store transaction that also completes the delivery. It must finish within the lease duration or its `timeout`. - After a timeout or lease loss the handler's `signal` is aborted; the retry waits for the abandoned run to settle so runs never overlap. ## Event catalogs A catalog groups the events of one domain so the compiler checks that every event has a handler and that only those events are published through it. ```ts class OrderPlaced extends defineEvent<{ orderId: string }>('order.placed.v1') {} class OrderShipped extends defineEvent<{ orderId: string; carrier: string }>('order.shipped.v1') {} export const OrderEvents = defineCatalog({ OrderPlaced, OrderShipped }, { name: 'OrderEvents' }); // A class that misses a handler does not compile: class ShippingHandlers implements HandlersOf { onOrderPlaced(event: EventOf, context: HandlerContext) {} onOrderShipped(event: EventOf) {} } bus.subscribeCatalog(OrderEvents, new ShippingHandlers()); // ids: OrderEvents.onOrderPlaced, ... // Or declare the handlers as an object; missing or extra keys do not compile: bus.subscribeCatalog( OrderEvents, defineHandlers(OrderEvents, { onOrderPlaced: (event) => reserveStock(event.payload.orderId), onOrderShipped: (event) => notify(event.payload.carrier), }), { idPrefix: 'fulfilment' }, ); // Publish through a view that only accepts the catalog's events: const orders = bus.forCatalog(OrderEvents); await orders.publish(new OrderPlaced({ orderId }), { transaction }); await orders.publish(new UserRegistered({ userId })); // compile error ``` - `defineCatalog(events, { name? })` returns a frozen catalog with frozen entries. It rejects empty catalogs, entries that are not event classes, names that are not identifiers of letters, digits and underscores (a name becomes part of a method name and a handler id) and two entries with one event type. - `HandlersOf` requires an `on${Key}` method per event; `HandlersOf` requires only those (`PartialHandlersOf` is the same type), and `HandlersOf` types `context.transaction`. `EventOf` is the envelope type of one event; `CatalogEvents` is the union of the catalog's (selected) events. - `bus.subscribeCatalog(catalog, handlers, { events?, idPrefix?, ...subscribeOptions })` registers the `on${Key}` method of every event (or of the `events` listed) with the stable id `${idPrefix ?? catalog name ?? class name}.on${Key}` and `this` bound to `handlers`. Methods of events that are not listed are ignored. A missing method does not compile and throws a `ConfigurationError` naming it. Handlers are narrowed only by an `events` list the compiler can see; options whose `events` may be absent require every handler. - `defineHandlers(catalog, handlers, { events? })` checks completeness, extra keys (including handlers of events that are not listed) and payload types at compile time and at runtime. A handler may type its context, for example `HandlerContext`. - **Boot check:** `new RelayBus({ catalogs: [OrderEvents] })` (or `bus.registerCatalog`) makes `start()` throw a `ConfigurationError` listing every catalog event that has no handler or route and the method it expects. A failed check leaves the bus idle, before any I/O, so it can be fixed and started again. For services that share a catalog, declare the role instead; see below. - `forCatalog(...).publish` also checks the event type at runtime. TypeScript compares event classes by shape, so two events with identical payload types are interchangeable to the compiler; the runtime check still rejects the one that is not in the catalog, before anything is stored. ### Microservices: shared catalogs In a system of services the catalog usually lives in a shared contracts package, and each service produces some of its events and consumes others. Declare what this service does with each part: ```ts import { consume, produce } from '@relaybus/core'; import { OrderEvents } from '@acme/contracts'; // orders service: publishes OrderPlaced to the broker, handles OrderShipped class DeliveryStatusHandlers implements HandlersOf { onOrderShipped(event: EventOf) {} } const bus = new RelayBus({ store, transports: [kafka], routes: [{ event: OrderPlaced, transport: 'kafka', destination: 'orders' }], subscriptions: [{ transport: 'kafka', destinations: ['shipments'] }], catalogs: [ produce(OrderEvents, { events: ['OrderPlaced'] }), consume(OrderEvents, { events: ['OrderShipped'] }), ], }); bus.subscribeCatalog(OrderEvents, new DeliveryStatusHandlers(), { events: ['OrderShipped'] }); const orders = bus.forCatalog(OrderEvents, { events: ['OrderPlaced'] }); await orders.publish(new OrderPlaced({ orderId }), { transaction, key: orderId }); await orders.publish(new OrderShipped({ orderId, carrier })); // compile error: not produced here // shipping service: handles OrderPlaced only class ShipmentHandlers implements HandlersOf { onOrderPlaced(event: EventOf) {} } ``` - `consume(catalog, { events? })`: each listed event (default: all) needs a **local handler**; a route does not count. `produce(catalog, { events? })`: each listed event needs a **route to a transport**, so nothing is published without a destination; a local handler does not count. Both can be declared for one catalog, and for one event. Unknown names do not compile and are rejected at runtime, as are empty and duplicate lists. A bare catalog keeps its meaning: every event needs a handler or a route. - `start()` lists every gap, per event: ```text ConfigurationError: Catalog events this service does not cover: - OrderEvents.OrderShipped (order.shipped.v1): consumed here, needs handler method onOrderShipped - OrderEvents.OrderPlaced (order.placed.v1): produced here, needs a route to a transport ``` - `bus.forCatalog(catalog, { events })` publishes only the listed events: others do not compile and are rejected at runtime before anything is stored. - Received events (`bus.receive`, called by Kafka and RabbitMQ consumers) are matched to the catalog handlers by event type, idempotently on the event id, and the payload is validated against the event's schema before the handler runs; a payload that fails it is dead-lettered. - Per-key order on the wire holds **per destination**: every route to one destination of one transport shares one queue per key, so `OrderPlaced` and `OrderCancelled` of one key reach the broker in publish order even when a send of the first is retried (Kafka: same partition; RabbitMQ: `ordered: true`). On the consuming side order holds **per handler**: two handlers of two types run independently and can interleave. ## Transports ```ts new RelayBus({ store, transports: [kafka], routes: [{ event: OrderPlaced, transport: 'kafka', destination: 'orders.events' }], subscriptions: [{ transport: 'kafka', destinations: ['payments.events'] }], }); ``` Routes are deliveries like any handler: retried until the broker accepts them (unlimited attempts by default) with a 30s publish timeout. Subscriptions call `bus.receive(envelope)`, which stores the event once by id for local handlers; transports acknowledge only after it resolves and dead-letter `InvalidEnvelopeError`. Every route to one destination of one transport is **one handler**, with the id `relaybus.transport::` (`handlers()` lists it once per routed event type). Its deliveries of one key are sent one at a time in commit order, whatever their event types: while the oldest is retried, the later ones of that key wait until it is sent or dead-lettered (a dead-lettered head releases them, as for any handler). Other keys and unkeyed events are not held back. Because the routes share one queue, they must resolve to the same `retry` and `timeout`; `new RelayBus` throws a `ConfigurationError` when they differ, and when one event type is routed to the same destination twice. Each delivery is one transport call: a hot key is sent one event per round trip (sending a run of one key's deliveries in one ordered call is a possible follow-up, not implemented). ### Upgrading from 0.2.0 0.2.0 registered one route per event type, with the id `relaybus.transport:::`, so events of one key but two types could reach the broker out of order. 0.3.0 groups them under `relaybus.transport::`. Nothing has to be migrated: deliveries stored under a 0.2.0 id, including those written by 0.2.0 processes during a rolling deploy, are still claimed and sent by the new route; the store rows keep their old id. - **Order during the upgrade.** A delivery under a 0.2.0 id and one under the new id are two queues, so one key's backlog from 0.2.0 is not ordered against events published after the upgrade. Where that matters, let the route backlog drain first (`listDeliveries({ handler: '<0.2.0 id>', status: 'pending' })` returns nothing) and stop 0.2.0 publishers before new ones start. - **Routes to one destination with different `retry` or `timeout`** now fail at construction; give them the same settings. - Code or dashboards that match route ids (`handlers()`, `listDeliveries({ handler })`, `HandlerContext.handler`, observer events) see the new id for new deliveries. ## Operations `listDeliveries(query)`, `requeue(ids)`, `statistics()`, `purgeExpired()`, `handlers()`, `runOnce()` and `drain()` (tests), `start()`, `stop()` and `wake()`. **`wake()` after your own commit.** A publish in relaybus' own transaction wakes the dispatcher right after its commit. A publish in your transaction commits at a time the bus cannot see: unless the store reports the commit (PostgreSQL with `LISTEN/NOTIFY`), the dispatcher is woken 25, 100 and 250 ms after the latest publish and otherwise finds the delivery at its next poll. Call `bus.wake()` right after such a commit to claim at once. **`stop({ timeout })` is bounded.** The whole shutdown (stop claiming, wait for running handlers, hand back leases, close transports and the store) takes at most `timeout`, default `dispatcher.shutdownTimeout` (30 s). Running handlers get four fifths of it; then their `context.signal` aborts and their deliveries are released for another process. If the deadline itself passes (a hung query, a broker that does not answer), `stop` gives up waiting, force-closes what relaybus opened itself (the pool a store created from `connectionString`, its listening connection, a RabbitMQ transport's connection) and resolves; it never closes a pool or client you passed in, and never rejects because of the deadline. It resolves with a `StopResult`: ```ts const { graceful, abandonedDeliveries, durationMs } = await bus.stop({ timeout: '25s' }); ``` `graceful` is false when the deadline passed, and a warning is logged (`[RelayBus](stop) Shutdown deadline passed, forced close | abandoned=1 | timeoutMs=25000`). Abandoned deliveries keep their lease until it expires (`dispatcher.leaseDuration`), then another process runs them again: at-least-once still holds, so keep handlers idempotent. Kafka's `disconnect` has no forced variant; past the deadline the bus simply stops waiting for it. **Observability.** Besides the lifecycle hooks, observers can implement timed hooks: `onPublishFinished` (event types, duration, outcome), `onClaimFinished` (requested, claimed, lanes, duration), `onHandlerStarted` (end-to-end latency from `occurredAt`), `onHandlerFinished` (duration, outcome, error with its code), `onTransportSent` (transport, system, destination, duration, outcome), `onReceiveFinished` (transport, destination, source, trace context; `accepted`, `duplicate`, `ignored`, `invalid` or `error`), `onInvalidMessage` (a transport could not decode an inbound message, so it never reached the bus: transport, destination, reason) and `onBacklog` (pending deliveries and age of the oldest due one, every `backlogInterval`). A hook no observer implements costs nothing, not even a clock read. Hooks run synchronously and must not throw; a throwing hook is reported to `onFailure` with stage `observer`. `traceContextOf(event.headers)` returns the W3C trace context an event carries. `@relaybus/otel` maps all of it to OpenTelemetry. **Completed deliveries are deleted.** Completing a delivery removes its row instead of updating it, which is what keeps the write cost of a delivery constant (see `docs/performance/write-amplification.md`). `listDeliveries({ status: 'completed' })` and `statistics().completed` therefore report nothing unless the store keeps a completion history (`completionHistory: true` on the SQL stores, `retainCompleted: true` on `InMemoryEventStore`). Deduplication does not depend on it: an event remembers every handler it ever had a delivery for, so a redelivered or republished event never runs a completed handler again, for as long as the event is kept (`retention.completed`). ## Error codes Every relaybus error has a stable, machine-readable `code` next to its `name`. Codes are never renamed or reused once released, so branch on them in alerts and retry rules. The `lastError` of a dead or retried delivery and the `error` of observer events carry the `code` of relaybus errors; other errors keep `name`, `message` and `stack` only. | Code | Error class | Package | | ------------------------------------------ | ------------------------------- | -------------------- | | `RELAYBUS_ERROR` | `RelayBusError` | `@relaybus/core` | | `RELAYBUS_CONFIGURATION` | `ConfigurationError` | `@relaybus/core` | | `RELAYBUS_TRANSACTION_REQUIRED` | `TransactionRequiredError` | `@relaybus/core` | | `RELAYBUS_TRANSACTION_TIMEOUT` | `TransactionTimeoutError` | `@relaybus/core` | | `RELAYBUS_NON_RETRYABLE` | `NonRetryableError` | `@relaybus/core` | | `RELAYBUS_EVENT_VALIDATION` | `EventValidationError` | `@relaybus/core` | | `RELAYBUS_INVALID_ENVELOPE` | `InvalidEnvelopeError` | `@relaybus/core` | | `RELAYBUS_LEASE_LOST` | `LeaseLostError` | `@relaybus/core` | | `RELAYBUS_HANDLER_TIMEOUT` | `HandlerTimeoutError` | `@relaybus/core` | | `RELAYBUS_DISPATCHER_STOPPED` | `DispatcherStoppedError` | `@relaybus/core` | | `RELAYBUS_ATTEMPTS_EXHAUSTED` | `AttemptsExhaustedError` | `@relaybus/core` | | `RELAYBUS_LIFECYCLE` | `LifecycleError` | `@relaybus/core` | | `RELAYBUS_DRAIN_INCOMPLETE` | `DrainIncompleteError` | `@relaybus/core` | | `RELAYBUS_KAFKA_INVALID_MESSAGE` | `InvalidKafkaMessageError` | `@relaybus/kafka` | | `RELAYBUS_KAFKA_UNENCODABLE_EVENT` | `UnencodableEventError` | `@relaybus/kafka` | | `RELAYBUS_KAFKA_PUBLISH_TIMEOUT` | `PublishTimeoutError` | `@relaybus/kafka` | | `RELAYBUS_KAFKA_HANDLER_FAILED` | `HandlerFailedError` | `@relaybus/kafka` | | `RELAYBUS_KAFKA_TRANSPORT_CLOSED` | `TransportClosedError` | `@relaybus/kafka` | | `RELAYBUS_KAFKA_GROUP_JOIN_TIMEOUT` | `GroupJoinTimeoutError` | `@relaybus/kafka` | | `RELAYBUS_POSTGRES_RECONCILE_FAILED` | `ReconcileFailedError` | `@relaybus/postgres` | | `RELAYBUS_RABBITMQ_UNAVAILABLE` | `RabbitMqUnavailableError` | `@relaybus/rabbitmq` | | `RELAYBUS_RABBITMQ_PUBLISH_REJECTED` | `RabbitMqPublishRejectedError` | `@relaybus/rabbitmq` | | `RELAYBUS_RABBITMQ_UNROUTABLE_MESSAGE` | `UnroutableMessageError` | `@relaybus/rabbitmq` | | `RELAYBUS_RABBITMQ_MALFORMED_MESSAGE` | `MalformedMessageError` | `@relaybus/rabbitmq` | | `RELAYBUS_RABBITMQ_UNSERIALIZABLE_PAYLOAD` | `UnserializablePayloadError` | `@relaybus/rabbitmq` | | `RELAYBUS_TESTING_CODEC_CONTRACT` | `CodecContractError` | `@relaybus/testing` | | `RELAYBUS_RABBITMQ_UNENCODABLE_EVENT` | `RabbitMqUnencodableEventError` | `@relaybus/rabbitmq` | | `RELAYBUS_RABBITMQ_AMQP_LIMIT_EXCEEDED` | `AmqpLimitExceededError` | `@relaybus/rabbitmq` | An error class you derive from `RelayBusError` inherits `RELAYBUS_ERROR` unless it sets its own `code`. See [Error codes](https://berkcansavur.github.io/relaybus/concepts/error-codes) for what each one means. ## Writing a store Implement `EventStore` and run `eventStoreContract` from [`@relaybus/testing`](https://berkcansavur.github.io/relaybus/packages/testing) against a real database. The contract defines fencing, ordering, commit-order sequencing, visibility and purge semantics. ## When to use @relaybus/core, and when not to Use it directly when you are not on NestJS, or when you write a custom `EventStore` or transport. Do not run it alone in production: add a store (`@relaybus/postgres` or `@relaybus/mongodb`). `InMemoryEventStore` refuses to start when `NODE_ENV=production`. Documentation: ### relaybus vs alternatives | | What it is | Survives a restart | Written in your database transaction | Choose it when | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | **relaybus** | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries | Yes, events live in your database | Yes | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys | | [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2` | No persistence is documented | No | Simple in-process notifications where losing an event on restart is acceptable | | [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs) | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory | No built-in persistence | No | You want CQRS structure; it can sit on top of an outbox | | [`@nestjs/outbox`](https://github.com/nestjs/outbox) | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores | Yes | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing | | [BullMQ](https://docs.bullmq.io/) | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case | Yes, in Redis | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity | | [pg-boss](https://github.com/timgit/pg-boss) | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction | Yes, in PostgreSQL | Yes, through its database adapters | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports | | Kafka client directly | A distributed log; the idempotent producer and transactions give exactly-once within Kafka | Yes, in Kafka | No, producing to Kafka is a second write | You consume and produce only inside Kafka; add an outbox when a database write must be announced | This table summarises the vendors' own documentation as of October 2026; check their docs for current details. Full comparison and sources: . ## License MIT --- Source: https://berkcansavur.github.io/relaybus/packages/nestjs # @relaybus/nestjs @relaybus/nestjs is the NestJS module of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: declare handlers with `@RelayBusConsumer` and publish inside your database transaction, and every committed event reaches every handler at least once. NestJS integration for [relaybus](https://berkcansavur.github.io/relaybus/packages/core): one module, one decorator, correct lifecycle. ## Install ```bash npm install @relaybus/core @relaybus/nestjs ``` Peer dependencies: `@nestjs/common` and `@nestjs/core` (`^11 || ^12`), `reflect-metadata`, `rxjs`. ## Quick start ```ts // app.module.ts @Module({ imports: [RelayBusModule.forRoot({ store: new PostgresEventStore({ pool }), source: 'orders' })], }) export class AppModule {} // shipping.handlers.ts - any provider of any module @Injectable() export class ShippingHandlers { @RelayBusConsumer(OrderPlaced) async onOrderPlaced(event: EventEnvelope<{ orderId: string }>) { /* ... */ } } // order.service.ts @Injectable() export class OrderService { constructor( private readonly bus: RelayBus, private readonly transactions: RelayBusTransactionHost, ) {} @RelayBusTransaction() async place(orderId: string): Promise { await this.transactions.current().query('INSERT INTO orders (id) VALUES ($1)', [orderId]); await this.bus.publish(new OrderPlaced({ orderId })); // joins the transaction } } ``` That is all: the module is global, the handler is discovered, the bus starts after bootstrap and stops on shutdown. Inject the bus anywhere with `constructor(private readonly bus: RelayBus) {}`; add the transaction type when you want it typed: `RelayBus` (the generic is erased at runtime, so DI is unaffected). Call `app.enableShutdownHooks()` in `main.ts` so SIGTERM/SIGINT close the app and let the bus drain running handlers. The whole `bus.stop()` is bounded by `dispatcher.shutdownTimeout` (default 30s): past it, running handlers are abandoned (their leases expire and another pod runs them again) and the connections relaybus opened itself are force-closed, so a hung query or broker cannot keep the pod alive until Kubernetes kills it. Keep the timeout below the pod's `terminationGracePeriodSeconds`, leaving room for the rest of `app.close()`: ```ts // main.ts const app = await NestFactory.create(AppModule); app.enableShutdownHooks(); await app.listen(3000); // app.module.ts, for a pod with terminationGracePeriodSeconds: 30 RelayBusModule.forRoot({ store, dispatcher: { shutdownTimeout: '25s' } }); ``` ## `RelayBusModule` - `forRoot(options)` - `forRootAsync({ imports?, inject?, useFactory })`, or `useClass` / `useExisting` (a class with `create(): RelayBusModuleOptions | Promise`, see `RelayBusModuleOptionsFactory`) Both accept `isGlobal` next to the options. It defaults to `true`; with `false`, import the module wherever `RelayBus` is injected. `RelayBus` and `RelayBusHealth` are exported. ### Options Everything of core's `RelayBusOptions` passes through unchanged, except `logger`: the bus always logs through a Nest `Logger` with context `RelayBus`, so `app.useLogger(...)` controls it. | Option | Default | Notes | | --------------------------------------------- | ------------------------- | -------------------------------------------------------------------------------- | | `store` | required | An `EventStore`: `@relaybus/postgres`, `@relaybus/mongodb`, `InMemoryEventStore` | | `source` | `'relaybus'` | Service name recorded on every event; set it to your app name | | `transactionMode` | `'required'` | `'optional'` lets `publish` open its own transaction | | `transactionResolver` | none | Finds the caller's active transaction (see below) | | `transactionTimeout` | `'5s'` | Budget of every `bus.transaction` / `@RelayBusTransaction()` (see below) | | `retry` | core default | Default retry policy of every handler and route | | `handlerTimeout` | `false` | Default handler timeout | | `dispatcher.enabled` | `true` | `false` on publish-only processes (API pods, scripts) | | `dispatcher.concurrency` | `10` | Handlers running at once | | `dispatcher.batchSize` | concurrency | Deliveries claimed per query | | `dispatcher.pollInterval` | `'1s'` | Wait between claims when idle | | `dispatcher.leaseDuration` | `'30s'` | Renewed every third of it | | `dispatcher.shutdownTimeout` | `'30s'` | Longest the whole shutdown takes; past it, handlers are abandoned | | `retention.completed` / `.dead` / `.interval` | `'7d'` / `false` / `'1h'` | Cleanup of finished deliveries | | `transports`, `routes`, `subscriptions` | none | Broker fan-out and fan-in (`@relaybus/kafka`, `@relaybus/rabbitmq`) | | `setupStore` | `true` | Create tables/indexes on start | | `observers` | none | Metrics and tracing hooks | | `clock`, `generateIdentifier`, `random` | system | Mainly for tests | | `isGlobal` | `true` | Module-level, see above | ## `@RelayBusConsumer(event, options?)` The default way to handle events: put `@RelayBusConsumer` on any method of any provider. ```ts @Injectable() export class ShippingHandlers { @RelayBusConsumer(OrderPlaced) async shipOrder(event: EventEnvelope<{ orderId: string }>) {} @RelayBusConsumer(UserRegistered, { id: 'shipping.welcome-pack', retry: { attempts: 5 } }) async sendWelcomePack(event: EventOf, context: HandlerContext) {} } ``` ```ts @RelayBusConsumer(OrderPlaced) // class @RelayBusConsumer('order.placed.v1', { concurrency: 2 }) // type string @RelayBusConsumer(OrderPlaced, { id: 'billing.invoice', transactional: true, timeout: '10s' }) ``` - **No class-level setup.** No catalog, interface or base class: method names are free, and one class may handle events of different catalogs (or of none). - **Typed at compile time.** With a class argument the first parameter must be the envelope of that event: `@RelayBusConsumer(OrderPlaced) ship(event: EventEnvelope<{ userId: number }>)` does not compile. - Options are core's `SubscribeOptions` with `id` optional. Methods receive `(envelope, context)`. **Names.** The relaybus decorators share the `RelayBus` prefix (`@RelayBusConsumer`, `@RelayBusCatalogConsumer`, `@RelayBusTransaction`), so they never collide with `@nestjs/event-emitter`'s `@OnEvent` or `@nestjs/cqrs`. `@Subscribe` and `@EventConsumer` are the earlier names of `@RelayBusConsumer` and `@RelayBusCatalogConsumer`: they still work, are the same functions, are marked `@deprecated` (editors strike them through) and are removed in 1.0.0. `@RelayBusConsumer` only compiles on a method and takes an event; `@RelayBusCatalogConsumer` only compiles on a class and takes a catalog. Rules, all enforced at boot (the application fails to start, no silent skips): - **Stable ids.** The default id is `ClassName.methodName`. Deliveries are stored under the id, so renaming the class or method orphans pending deliveries; pin `{ id }` on handlers that matter. - **Duplicate ids** fail init with both locations named. A method with several `@RelayBusConsumer` decorators must give each an explicit `id`. - **Singletons only.** Handlers on request-scoped or transient providers (or default-scoped providers that depend on one) are rejected: deliveries run outside any request, so there is no request instance to call. This covers class, `useClass` and `useFactory` providers (a factory is checked through its class token; a request-scoped factory under a string token cannot be inspected). Providers without `@RelayBusConsumer` are never inspected for scope. - `this` is bound to the provider instance. Only providers are scanned, in every module. A handler reachable through several tokens (`useExisting`) is registered once. Declarations are stored per class and method, so decorators that wrap the method later do not drop them, and subclasses inherit the handlers of their base class. ## Module contracts: `RelayBusModule.forFeature(...)` The recommended way to use catalogs in NestJS. A feature module states which events it consumes and produces; the method-level `@RelayBusConsumer` handlers of its providers fulfil the contract, spread over any classes and method names. ```ts @Module({ imports: [ RelayBusModule.forFeature({ consumes: [OrderEvents], // or consume(OrderEvents, { events: ['OrderPlaced'] }) produces: [produce(BillingEvents, { events: ['InvoiceIssued'] })], }), ], providers: [ChargeHandlers, RefundHandlers], }) export class BillingModule {} ``` When the application initialises: - every consumed event needs a `@RelayBusConsumer` method (or a `@RelayBusCatalogConsumer` class) among the providers **of that module**; consumers in other modules do not count; - every produced event needs a route to a transport (`routes` of `forRoot`). Otherwise the application fails to start with one line per gap, for example `[RelayBusHandlerExplorer](assertModuleContracts) BillingModule: OrderEvents.OrderCancelled (order.cancelled.v1) has no consumer`. `forRoot({ catalogs })` keeps working as the application-wide check (handlers anywhere). ## Stricter checking (optional): catalogs and `@RelayBusCatalogConsumer(catalog, options?)` Method-level handlers are enough for most services. When a team wants the compiler and the boot to prove that _every_ event of a shared contract is handled, group the events in a catalog and let a class consume it. Everything here is optional and can be added later, next to `@RelayBusConsumer` methods. ```ts export const OrderEvents = defineCatalog({ OrderPlaced, OrderShipped }, { name: 'OrderEvents' }); @Injectable() @RelayBusCatalogConsumer(OrderEvents, { transactional: true }) export class ShippingHandlers implements HandlersOf { onOrderPlaced(event: EventOf, context: HandlerContext) {} onOrderShipped(event: EventOf) {} } RelayBusModule.forRoot({ store, catalogs: [OrderEvents] }); ``` Registers every `on${EventName}` method the class defines, under the stable id `ClassName.on${EventName}`, with `this` bound to the provider. `implements HandlersOf<...>` makes the compiler reject a class that misses a handler; `PartialHandlersOf` documents a deliberate subset. Options are core's `SubscribeOptions` without `id`, applied to every handler of the class. - `catalogs: [OrderEvents]` in the module options is the boot check: if no provider (via `@RelayBusCatalogConsumer`, `@RelayBusConsumer` or a configured route) handles an event, the application fails to start with the list of missing events and the methods they expect. - It works next to `@RelayBusConsumer` in the same class. A `@RelayBusConsumer` that would take the same default id (`@RelayBusConsumer(OrderPlaced) onOrderPlaced()`) fails init naming both; give it an explicit `id`. - Same rules as `@RelayBusConsumer`: singleton providers only, `useExisting` aliases register once, subclasses inherit the decoration and use their own class name in ids. - A class decorated with `@RelayBusCatalogConsumer` that defines none of the catalog's methods fails init, which catches misspelled method names. - Declarations are inherited from base classes; a subclass that declares the same catalog again replaces the inherited declaration, and one catalog declared twice on one class throws. ### Microservices: consuming part of a shared catalog ```ts // shipping service @Injectable() @RelayBusCatalogConsumer(OrderEvents, { events: ['OrderPlaced'] }) export class ShippingHandlers { onOrderPlaced(event: EventOf) {} } RelayBusModule.forRoot({ store, transports: [rabbit], subscriptions: [{ transport: 'rabbit', destinations: ['orders'] }], routes: [{ event: OrderShipped, transport: 'rabbit', destination: 'shipments' }], catalogs: [ consume(OrderEvents, { events: ['OrderPlaced'] }), produce(OrderEvents, { events: ['OrderShipped'] }), ], }); ``` - With `events`, exactly those methods are registered (other `on${EventName}` methods are ignored), and a class that misses one of them does not compile and fails init. - `consume(...)` fails the boot when a consumed event has no handler in the application; `produce(...)` fails it when a produced event has no route. See `@relaybus/core`, "Microservices: shared catalogs". ## Lifecycle 1. `onModuleInit`: handlers are discovered and registered on the bus. 2. `onApplicationBootstrap`: `bus.start()` creates the schema (unless `setupStore: false`), connects transports and starts the dispatcher. If it rejects (database down, broker unreachable) the application does not boot. 3. `beforeApplicationShutdown`: `bus.stop()` stops claiming, waits for running handlers, then closes transports and the store, all within `shutdownTimeout` (it force-closes and resolves when the deadline passes, see above). Needs `app.enableShutdownHooks()` for signals; `app.close()` runs it regardless. Nest runs `onApplicationShutdown` hooks (where TypeORM, Mongoose and most pools disconnect) after this, so handlers still have their connections while draining. If your own provider disconnects in `onModuleDestroy` (common with Prisma), move it to `onApplicationShutdown`. ## Transactions ### `@RelayBusTransaction()` Runs an async method in a relaybus transaction. Every `bus.publish(...)` inside it, and inside everything it awaits, joins the transaction without `{ transaction }`; the transaction commits when the method resolves and rolls back when it throws. ```ts // Before: the transaction is threaded through by hand async place(orderId: string): Promise { await this.bus.transaction(async (tx) => { await tx.query('INSERT INTO orders (id) VALUES ($1)', [orderId]); await this.bus.publish(new OrderPlaced({ orderId }), { transaction: tx }); }); } // After @RelayBusTransaction() async place(orderId: string): Promise { await this.orders.insert(orderId); // the repository reads RelayBusTransactionHost.current() await this.bus.publish(new OrderPlaced({ orderId })); } ``` - **Self-calls are transactional.** The decorator replaces the method itself (not an interceptor or proxy), so `this.place()` from another method of the class opens the transaction too. - **Nested calls share one transaction.** A decorated method called inside another one, inside `bus.transaction`, inside a transactional handler, or while `transactionResolver` finds a transaction (for example `@nestjs-cls/transactional`) joins it instead of opening a second one. - **No injection needed.** The bus is the one of the application's `RelayBusModule`. Calling the method before the application initialised, after it shut down, or with no `RelayBusModule` throws a `ConfigurationError` that says so. - **Async methods only.** On a method that does not return a Promise it fails to compile: `'@RelayBusTransaction() needs a method that returns a Promise: make the method async'`. ### Isolation levels: `@RelayBusTransaction({ isolationLevel })` `@RelayBusTransaction({ isolationLevel: 'serializable', maxAttempts: 5 })` (and `bus.transaction(work, { isolationLevel, maxAttempts })`) runs the method at `'read committed'`, `'repeatable read'` or `'serializable'` (PostgreSQL stores; MongoDB and the in-memory store reject a level with a `ConfigurationError`). At `'repeatable read'` and `'serializable'` a serialization failure (SQLSTATE `40001`) rolls the attempt back and **runs the method again** in a new transaction, up to `maxAttempts` attempts in all (default 3); the last failure is thrown unchanged. That lets PostgreSQL detect write skew (count the active rows, check a limit, insert) without a `pg_advisory_xact_lock` or `SELECT ... FOR UPDATE`. ```ts @RelayBusTransaction({ isolationLevel: 'serializable' }) async reserve(userId: string): Promise { if ((await this.reservations.countActive(userId)) >= LIMIT) throw new LimitReachedError(userId); await this.reservations.insert(userId); await this.bus.publish(new ReservationCreated({ userId })); } ``` - **The method may run more than once.** Keep HTTP calls, broker publishes and other side effects that are not database writes out of it. Events published by a failed attempt are rolled back with it, and `onPublished` and the dispatcher wake-up wait for the attempt that committed. - **Joining.** A method with an `isolationLevel` joins a running transaction only when relaybus opened it at the same level; otherwise it throws `TransactionIsolationConflictError` (`RELAYBUS_TRANSACTION_ISOLATION_CONFLICT`), because the level of a running transaction cannot be changed. A method without `isolationLevel` joins any transaction, as before. - Without `isolationLevel` nothing differs from `@RelayBusTransaction()`: same statements, no rerun. See [Serializable transactions](https://berkcansavur.github.io/relaybus/recipes/serializable-transactions). ### Time budget: `transactionTimeout` Every relaybus transaction (`bus.transaction` and `@RelayBusTransaction()`) is held to `transactionTimeout` (default `'5s'`). The database enforces it where it can, so a transaction waiting on something slow is ended by the server and its row locks are released: - **PostgreSQL:** `SET LOCAL statement_timeout`, and with the `pg` executor also `idle_in_transaction_session_timeout` and, on PostgreSQL 17+, `transaction_timeout` (detected per transaction). Adapters other than `pg` get `statement_timeout` only, because a server-ended session would leave their checked-out connection without an `error` listener. - **MongoDB:** `maxCommitTimeMS` on the commit. A transaction that still runs past the budget is rolled back before it commits. Either way the caller gets a `TransactionTimeoutError` (`RELAYBUS_TRANSACTION_TIMEOUT`, with the database error as `cause`) whose message points at the usual cause: a slow call to another service inside the transaction. Move HTTP calls before the transaction or after it commits (the `@relaybus/eslint-plugin` rule `no-external-call-in-transaction` catches them) instead of raising the budget. ### Repositories: `RelayBusTransactionHost` Inject `RelayBusTransactionHost` typed with your store's transaction handle. `current()` returns the active transaction and throws `TransactionRequiredError` outside one; `find()` returns `undefined` instead. Outside Nest, `bus.currentTransaction()` does the same as `find()`. | Store / executor | `T` of `RelayBusTransactionHost` | Write with | | ----------------------------------- | ----------------------------------- | -------------------------------------------- | | `PostgresEventStore` with `pg` | `PoolClient` | `tx.query(sql, values)` | | `createKyselyExecutor(db)` | `Transaction` | `tx.insertInto('orders')...execute()` | | `createTypeOrmExecutor(dataSource)` | `EntityManager` | `tx.getRepository(Order).save(order)` | | `createPrismaExecutor(prisma)` | `Prisma.TransactionClient` | `tx.order.create({ data })` | | `MongoEventStore` | `ClientSession` | `collection.insertOne(doc, { session: tx })` | Without a decorator or `bus.transaction`, `publish` needs a transaction: pass `{ transaction }`, configure `transactionResolver`, or set `transactionMode: 'optional'` (event stays durable but is no longer atomic with your writes). ### Transactional handlers ```ts @RelayBusConsumer(PaymentCaptured, { id: 'ledger.record-capture', transactional: true }) async record(event: EventEnvelope<{ paymentId: string }>, context: HandlerContext) { // Writes through context.transaction commit atomically with the delivery completion: // effectively-once processing. Keep these handlers short. await context.transaction!.query('INSERT INTO ledger ...', [event.payload.paymentId]); await this.bus.publish(new LedgerUpdated({ ... }), { transaction: context.transaction! }); } ``` Non-transactional handlers are at-least-once: make them idempotent (deduplicate on `event.id`). ### ORM integration Prefer `transactionResolver`, so services call `bus.publish(event)` inside whatever transaction the ORM already opened. The resolver is called synchronously and must return `undefined` when there is no active transaction. ```ts // @nestjs-cls/transactional (not a dependency of this package) RelayBusModule.forRootAsync({ inject: [TransactionHost], useFactory: (txHost: TransactionHost) => ({ store, transactionResolver: () => (txHost.isTransactionActive() ? txHost.tx : undefined), }), }); ``` - **Prisma / raw `pg`:** `@relaybus/postgres` store; Prisma's interactive transaction client is adapted by the store package, see its README. - **TypeORM:** `@relaybus/postgres` over the same `DataSource` connection; resolve the active `EntityManager`'s query runner in `transactionResolver`. - **MongoDB:** `@relaybus/mongodb` store; the transaction is the `ClientSession` (needs a replica set). ## Health `RelayBusHealth` (exported by the module) has no `@nestjs/terminus` dependency. ```ts const report = await health.check(); // { status: 'up' | 'down', running, ready, handlers, statistics?, lagMilliseconds?, error? } ``` `down` when the bus is not running, a transport is not ready (`ready` is `bus.isReady()`: for example a dead Kafka consumer or a disconnected RabbitMQ channel) or the store cannot be queried. `lagMilliseconds` is how long the oldest due delivery has waited (measured with the `clock` from the module options, the system clock by default), a good alert signal. With terminus 11: ```ts @Injectable() export class RelayBusIndicator { constructor( private readonly indicators: HealthIndicatorService, private readonly relayBus: RelayBusHealth, ) {} async isHealthy(key = 'relaybus') { const indicator = this.indicators.check(key); const { status, ...details } = await this.relayBus.check(); return status === 'up' ? indicator.up(details) : indicator.down(details); } } ``` ## Startup failure and restarts The bus starts during application bootstrap. If the start fails (store setup, transport connect), application boot fails with that error. A bus whose start failed is stopped for good and cannot be started again: restart the process rather than retrying in place. ## Testing Use `@nestjs/testing` with `InMemoryEventStore` and turn the dispatcher off so tests control time: ```ts const moduleRef = await Test.createTestingModule({ imports: [ RelayBusModule.forRoot({ store: new InMemoryEventStore(), dispatcher: { enabled: false } }), AppFeatureModule, ], }).compile(); await moduleRef.init(); await moduleRef.get(OrderService).place('o-1'); await moduleRef.get(RelayBus).drain(); // runs every due delivery, including handlers' follow-up events // assert on side effects await moduleRef.close(); ``` - `bus.runOnce()` runs a single batch; `bus.drain()` loops until nothing is due. - Retries and delays are scheduled in the future: pass `clock: new FakeClock()` from `@relaybus/testing`, `clock.advance(5_000)`, then `drain()` again. - `bus.listDeliveries({ status: 'dead' })` shows handlers that exhausted their retries. ## Troubleshooting - **`Nest can't resolve dependencies of ... (?)`** when injecting `RelayBus`: the module is not global (`isGlobal: false`) and not imported by the consuming module. - **Handler never runs:** the provider is not listed in any module `providers`, the process runs with `dispatcher.enabled: false`, or the event type differs (check `bus.handlers()`). - **`uses @RelayBusConsumer but its provider is request-scoped or transient`:** make the provider a singleton; inject request data through the event payload instead. - **Vitest/esbuild:** esbuild does not emit decorator metadata; use explicit `@Inject(Token)` in test classes. - **App hangs on shutdown:** a handler ignores `context.signal`; it is waited for until `dispatcher.shutdownTimeout`. ## When to use @relaybus/nestjs, and when not to Use it when your service is a NestJS 11 or 12 application and you want decorator-based handlers, lifecycle integration and a health check. It needs `@relaybus/core` and a store; for other frameworks use `@relaybus/core` directly. Documentation: ### relaybus vs alternatives | | What it is | Survives a restart | Written in your database transaction | Choose it when | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | **relaybus** | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries | Yes, events live in your database | Yes | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys | | [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2` | No persistence is documented | No | Simple in-process notifications where losing an event on restart is acceptable | | [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs) | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory | No built-in persistence | No | You want CQRS structure; it can sit on top of an outbox | | [`@nestjs/outbox`](https://github.com/nestjs/outbox) | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores | Yes | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing | | [BullMQ](https://docs.bullmq.io/) | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case | Yes, in Redis | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity | | [pg-boss](https://github.com/timgit/pg-boss) | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction | Yes, in PostgreSQL | Yes, through its database adapters | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports | | Kafka client directly | A distributed log; the idempotent producer and transactions give exactly-once within Kafka | Yes, in Kafka | No, producing to Kafka is a second write | You consume and produce only inside Kafka; add an outbox when a database write must be announced | This table summarises the vendors' own documentation as of October 2026; check their docs for current details. Full comparison and sources: . --- Source: https://berkcansavur.github.io/relaybus/packages/postgres # @relaybus/postgres @relaybus/postgres is the PostgreSQL event store of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: it writes the transactional outbox in your own transaction, with `pg`, Prisma, TypeORM, Drizzle, Knex or Kysely. PostgreSQL event store for [relaybus](https://github.com/berkcansavur/relaybus): a transactional outbox with lane leases, batched claim and completion, per-key ordering and fenced state changes. PostgreSQL 13 or newer. Works with `pg`, Kysely, Knex, TypeORM, Drizzle and Prisma, so events are written in the **same transaction as your business data**. ## Install ```bash npm install @relaybus/core @relaybus/postgres pg npm install -D @types/pg # plus the library you already use (optional peers): kysely | knex | typeorm | drizzle-orm ``` ## Usage ### `pg` (default) ```ts import { Pool } from 'pg'; import { RelayBus } from '@relaybus/core'; import { PostgresEventStore } from '@relaybus/postgres'; const pool = new Pool({ connectionString: process.env.DATABASE_URL }); const bus = new RelayBus({ store: new PostgresEventStore({ pool }) }); bus.subscribe( OrderPlaced, async (event) => { /* ... */ }, { id: 'send-confirmation' }, ); await bus.start(); // creates tables unless setupStore: false const client = await pool.connect(); try { await client.query('BEGIN'); await client.query('INSERT INTO orders ...'); await bus.publish(new OrderPlaced({ orderId }), { transaction: client, key: orderId }); await client.query('COMMIT'); } catch (error) { await client.query('ROLLBACK'); throw error; } finally { client.release(); } ``` The transaction handle is a `pg` `PoolClient`. If you pass `connectionString` instead of `pool`, the store creates and, on `close()`, ends its own pool. A pool or executor you pass in is never closed by the store. When `bus.stop()` runs past its deadline, the store destroys the connections it opened itself (its own pool's and the one it listens for notifications on), so a query stuck on a lock fails at once instead of holding the shutdown; your pool is left as is. ### Other libraries Each adapter lives in its own entry point, so `@relaybus/postgres` itself never imports an ORM. | Library | Import | Transaction handle you pass to `publish` | | ------------------------- | --------------------------------------------------------- | --------------------------------------------------------------- | | Kysely | `createKyselyExecutor` from `@relaybus/postgres/kysely` | `trx` of `db.transaction().execute(trx => ...)` | | Knex | `createKnexExecutor` from `@relaybus/postgres/knex` | `trx` of `knex.transaction(trx => ...)` | | TypeORM | `createTypeOrmExecutor` from `@relaybus/postgres/typeorm` | the `EntityManager` of `dataSource.transaction(manager => ...)` | | Drizzle (`node-postgres`) | `createDrizzleExecutor` from `@relaybus/postgres/drizzle` | `tx` of `db.transaction(tx => ...)` | | Prisma | `createPrismaExecutor` from `@relaybus/postgres/prisma` | `tx` of `prisma.$transaction(async tx => ...)` | ```ts import { createKyselyExecutor } from '@relaybus/postgres/kysely'; const store = new PostgresEventStore({ executor: createKyselyExecutor(db) }); const bus = new RelayBus({ store }); await db.transaction().execute(async (trx) => { await trx.insertInto('orders').values(order).execute(); await bus.publish(new OrderPlaced({ orderId: order.id }), { transaction: trx }); }); ``` Use `transactionResolver` (for example from `@nestjs-cls/transactional`) to avoid passing the handle explicitly. For another client, implement `PostgresExecutor` (`transaction` and `query`; about 20 lines). **Transaction time budget.** `bus.transaction` passes the bus `transactionTimeout` (default `'5s'`) to the store, which sets it with `set_config(..., true)` (the `SET LOCAL` scope, gone after the transaction) as `statement_timeout`. Executors that declare `sessionTimeouts: true` (`PgExecutor` does; it listens for `error` on the connection while the transaction runs) also get `idle_in_transaction_session_timeout` and, on PostgreSQL 17+, `transaction_timeout`, so a transaction left idle by a slow external call is ended by the server and its locks are released. The other adapters keep only `statement_timeout`, because a server-ended session would emit an unhandled `error` on their checked-out `pg` connection; the bus still rolls back an overrun before the commit. The caller gets a `TransactionTimeoutError` with the server error as `cause`. `PgExecutor` sends the settings in the round trip of `BEGIN` (the optional executor method `transactionWithBudget`), so the budget costs no extra round trip; executors without it send them as the first statement of the transaction. **Isolation levels.** `bus.transaction(work, { isolationLevel })` opens the transaction at `'read committed'`, `'repeatable read'` or `'serializable'`, and at the last two reruns `work` after a serialization failure (SQLSTATE `40001`; see the `@relaybus/nestjs` README). `PgExecutor` sends `BEGIN ISOLATION LEVEL ...` in the same round trip as the budget settings, so a level costs no statement. The TypeORM adapter uses TypeORM's own `transaction('SERIALIZABLE', ...)` (so the level is set before TypeORM subscribers run). Any other executor gets `SET TRANSACTION ISOLATION LEVEL ...` as the first statement of the transaction and the budget statement second (PostgreSQL refuses the level once a query ran, and the budget's `set_config` is a query). The store recognizes `40001` through `pg` (also what Kysely and Knex pass through), TypeORM's `driverError`, the `cause` of a Drizzle error and Prisma's `P2034`, `meta.code` and driver-adapter errors. A custom executor implements the optional `transactionWithIsolation` to send the level with its `BEGIN`, and the optional `isSerializationFailure` for an error shape the store does not know. Where the budget lives depends on who owns the pool: - **A pool the store created** (`connectionString`/`poolConfig`): each connection keeps the budget as a session setting. The first `bus.transaction` on a connection sets it with `BEGIN`; later ones send nothing extra, so a publish costs what it did without a budget. A transaction without a budget (setup, a transactional handler) puts the session defaults back in its `BEGIN`. Statements outside a transaction (claims, completions) run under the budget the connection last got. With `preparedStatements: false` (a pooler in transaction mode, which would carry session settings to other clients) the budget stays transaction-scoped. - **A pool or executor you pass in**: the budget is transaction-scoped (`SET LOCAL` semantics), set with `BEGIN` by `PgExecutor` and by its own statement for the other adapters. Session settings of your pool are never changed. **Prisma** 5, 6 and 7 are supported (optional peer `@prisma/client` `^5.0.0 || ^6.0.0 || ^7.0.0`). It is integration-tested with a generated Prisma Client 7.10 (and 6.19 and 5.22 in CI) on the `@prisma/adapter-pg` driver adapter: the full event store contract, publishing in `prisma.$transaction`, rollback and concurrent transactions. The adapter is typed structurally, so the package has no Prisma dependency. Prisma interactive transactions default to a 5 s timeout, so pass `createPrismaExecutor(prisma, { timeout, maxWait })` if handlers run longer. **Give an ORM store the `pg` pool too.** ORMs send raw statements unprepared and offer no connection to `LISTEN` on. Pass the `pg` pool your ORM's driver uses as `pool` next to `executor`: the statements the store sends outside your transactions (the dispatcher's claims, completions and renewals) then run on it as prepared statements, and the store listens for wake-ups, so an event published in your ORM transaction is dispatched right after that transaction commits. Your transactions still go through the ORM, and the store never closes the pool. ```ts const pool = new Pool({ connectionString }); const prisma = new PrismaClient({ adapter: new PrismaPg(pool) }); const store = new PostgresEventStore({ executor: createPrismaExecutor(prisma), pool }); ``` ## Options | Option | Default | Meaning | | ---------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------ | | `pool` / `connectionString` / `executor` | required, exactly one | How to reach the database. With `executor`, `pool` may be given too (see above). | | `poolConfig` | none | Extra `pg` pool settings when `connectionString` is used. | | `schema` | `'public'` | Schema of the tables. Created by `setup()` when it is not `public`. | | `tablePrefix` | `'relaybus_'` | Tables are `events`, `deliveries`, `key_sequences`, `lanes`. | | `completionHistory` | `false` | Keep an append-only record of completed deliveries (`completed_deliveries`). Same value in every process. | | `purgeBatchSize` | `1000` | Deliveries deleted per transaction by `purge`. | | `logger` | `console` | Receives idle-connection errors of a pool the store created itself. | | `preparedStatements` | `true` | Prepare the statements the store repeats once per connection. `false` behind PgBouncer < 1.21 in transaction mode. | | `wakeUp` | `'notify'` | `'notify'`: coalesced `NOTIFY` + `LISTEN` wake other processes. `'poll'`: no notifications, other processes poll. | **Pool errors.** A pool created from `connectionString` gets an `error` listener that logs through `logger` (`[PostgresEventStore](connect) Idle connection failed ...`); `pg` then replaces the dead connection, so a failover or `pg_terminate_backend` of an idle connection does not crash the process. A `pool` you pass in is yours: attach `pool.on('error', ...)` to it yourself, because without a listener `pg` re-throws the error as an uncaught exception. `schema` and `tablePrefix` are validated (`[a-z_][a-z0-9_]*`, names at most 63 characters) before they are placed in SQL; anything else throws `ConfigurationError`. ## Schema and migrations `setup()` (called by `bus.start()`) creates everything with `IF NOT EXISTS` inside a transaction guarded by `pg_advisory_xact_lock`, so any number of processes can start at once. The lock key is derived from the schema name, so stores with different table prefixes in one schema (which race on `CREATE SCHEMA IF NOT EXISTS`) are serialized too. To own the schema yourself, set `setupStore: false` and put the DDL into your migration tool: ```ts import { createPostgresSchemaSql, createPostgresSchemaStatements } from '@relaybus/postgres'; const migration = createPostgresSchemaSql({ schema: 'app', tablePrefix: 'bus_' }); ``` | Table | Purpose | | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `events` | One row per event: `id` (**text**, so any caller id works), `type`, `source`, `key`, `sequence`, `payload json`, `headers json`, correlation and causation ids, `occurred_at`, `created_at`, and `handlers`, the handlers that ever got a delivery (it outlives the deliveries). | | `deliveries` | One row per (event, handler) that is not finished: `lane`, `status` (`pending` or `dead`), `attempts`, `available_at`, `last_error jsonb`, key and sequence copied from the event, timestamps. `unique (event_id, handler)`, foreign key to events with `ON DELETE CASCADE`. | | `lanes` | 256 rows, one per lane: `epoch` (the fencing token), `lease_owner`, `lease_expires_at` and `in_flight` (ids of the deliveries the owner claimed). `fillfactor = 50` and no secondary index, so its updates are heap-only (HOT). | | `event_completions` | One row per event that had a delivery complete, with the time of its last completion. Retention of completed work (`retention.completed`) is measured from it, however old the event is; events that never had a delivery complete are measured from their creation. | | `key_sequences` | One counter row per ordering key. | | `completed_deliveries` | Only with `completionHistory`: one insert-only row per completed delivery. | A delivery row is inserted once and deleted once when it completes. Claiming, renewing a lease and completing never update it: they update the lane row (claim, renewal) or delete the delivery (completion), so the indexed columns of a delivery are written exactly once, at publish. Indexes on `deliveries`: `(lane, available_at, id) WHERE status = 'pending'` (claim and statistics), `(lane, handler, key, sequence) WHERE key IS NOT NULL AND status = 'pending'` (ordering check and the blocked-key claim), the unique `(event_id, handler)`, and partial indexes for purging dead deliveries (and completed ones left by 0.1.0); on `events`, `(created_at, id)` for the purge sweep. `listDeliveries` has no index of its own: it orders by `created_at`, so on a large `deliveries` table combine it with `status`/`handler`, which keeps the scan to the dead or pending rows, or add the index you need. `deliveries.id` uses the `C` collation so id tie-breaks are the same on every database. The tables are created with aggressive autovacuum settings (`autovacuum_vacuum_scale_factor = 0.02`): deleted deliveries leave dead tuples that autovacuum reclaims. ## Upgrading from 0.1.0 The step-by-step guide, including `completionHistory` and PgBouncer settings, is [Upgrading from 0.1 to 0.2](https://berkcansavur.github.io/relaybus/upgrading). `setup()` upgrades a 0.1.0 schema in place, in one transaction under the same advisory lock, and is safe to run again: it adds `lane` to every delivery (hash of the key, or of the event id) and `handlers` to every event (from the existing deliveries), turns `processing` deliveries into `pending` (their attempt stays counted, as a lost lease would count it), drops the per-delivery lease columns and the indexes that no longer exist, and creates the lane table. Deliveries that were `completed` stay as `completed` rows until `retention.completed` purges them (or forever with `retention: { completed: false }`). **Stop every 0.1.0 worker before running it** (they cannot use the new tables), and expect the one-off rewrite of `deliveries` (an `UPDATE` of every row plus `SET NOT NULL`) to take time proportional to its size; purge old completed rows first to shorten it. If you apply `createPostgresSchemaSql()` with your own migration tool, run it as one transaction with the workers stopped. Running 0.1.0 and the new version together is not supported. ## Guarantees - **Atomic publish.** Events and deliveries are inserted in your transaction; roll back and nothing exists. - **Per-key order.** On append, a key's counter row in `key_sequences` is upserted (`INSERT ... ON CONFLICT (key) DO UPDATE SET sequence = sequence + n`). That row lock is held until the publishing transaction ends, so two transactions publishing the same key are serialized and the sequence order equals the commit order. Within one `append` (one `publishAll`) keys are locked in sorted order and events and deliveries are inserted in primary-key order, so concurrent single calls cannot deadlock each other. Claiming hands out a keyed delivery only when no earlier (same handler and key) delivery is pending or processing, so one key is processed one event at a time, in order. A retrying or leased head blocks its successors; a dead-lettered head releases them. A `limit` of zero or less claims nothing. - **Lane leases.** Every delivery belongs to one of 256 lanes: a hash of its ordering key, or of its event id when it has none, so all deliveries of a key share a lane. Workers lease lanes, not rows. A claim runs in one short transaction: it locks the lane rows this worker already holds, takes free lanes that have due work (four, then eight, ... lanes at a time, so a worker holds only as many lanes as its batch needs, oldest due delivery first), reads the due deliveries of those lanes and writes **one** lane row per lane to record the batch (`in_flight`). It writes no delivery row and takes no row lock on one: exclusivity comes from the lease. A lane taken from another worker gets a new `epoch`; lanes that yielded nothing are handed back at once. While a lane lease is live, other workers do not see the pending deliveries of that lane. The lease is renewed with the running deliveries (`extendLease` updates the lane row) and handed back at graceful shutdown (`relinquish`). - **Per-key order under lanes.** A keyed delivery is claimed only when no earlier delivery of its handler and key is pending (a running one is still a pending row) and none of its handler and key is in flight, and the lane belongs to one worker, so a key is processed one event at a time, in order. A retrying or running head blocks its successors; a dead-lettered head releases them. Claiming first looks at the 256 earliest due deliveries of the lanes it holds (cheap); only when those are mostly followers of blocked keys and fewer than `limit` were picked does a second statement find the head of every key with `DISTINCT ON (handler, key)`. The stress test in `test/claim-concurrency.integration.test.ts` runs 12 claimers on 3 pools against 30 keys and fails if a key is ever in flight twice or out of order. - **Fenced.** A lease token is `::`, where `claims` counts how many claims of that delivery ended before this one (every `reschedule`, `release`, `bury` and takeover bumps it), so a lease of an older attempt never matches a newer claim even when the same worker re-claims the delivery in the same lane epoch. `complete`, `completeMany`, `reschedule`, `bury`, `release` and `extendLease` apply only while the lane still has that epoch and lists the delivery as in flight; a worker that lost its lane changes nothing. Completion takes a `FOR KEY SHARE` lock on the lane row until the transaction ends, so a takeover (which locks the row `FOR UPDATE`) and a transactional handler's completion cannot both succeed: the handler's writes and the deletion of its delivery commit together, or the takeover redelivers it. - **Attempts survive crashes.** An attempt is recorded when it ends: a failure adds one to `attempts` (`reschedule`, `bury`), a graceful `release` adds nothing, and a lane that is taken over charges one attempt to every delivery its previous holder still had in flight. A worker that dies on a poison message therefore still runs it into the retry limit and the dead-letter table. - **No double run after completion.** An event lists every handler it ever had a delivery for, so republishing an event id (or a broker redelivering it) creates deliveries only for handlers that never had one, even though completed deliveries are deleted. This holds while the event row exists (`retention.completed`). - **App clock.** Every timestamp is the `now` the caller passes, never the database clock. - **Driver independent reads.** Timestamps and JSON are read as text and parsed by the store, so custom `pg` type parsers (or Drizzle's) cannot change results. ## Operations - **Isolation.** Claiming runs as a short transaction and relies on `READ COMMITTED` (the PostgreSQL default). Do not change `default_transaction_isolation` for the relaybus pool. Your own publishing transaction may use a stricter level, but under `REPEATABLE READ` or `SERIALIZABLE` PostgreSQL can abort it with `40001` (serialization failure); retry the whole transaction. - **Deadlocks.** Publishing a keyed event locks that key's counter row until your transaction ends. Two transactions that append keys in opposite order across **several** `append`/`publish` calls (A then B in one, B then A in the other) can deadlock; PostgreSQL aborts one with `40P01`. The aborted transaction rolls back completely, nothing is lost or half-written, and the caller should retry it. Publishing all keys of a transaction in one `publishAll` call avoids the problem because the store locks them in sorted order. - **Retention.** Completed deliveries are deleted when they complete, so `purge` has two jobs: it deletes dead (`updated_at < cutoff`) deliveries, completion records (`completionHistory`) and `completed` rows left by 0.1.0 in batches of `purgeBatchSize`, each batch in its own short transaction with `SKIP LOCKED`, and it deletes the events left without deliveries or records, including events created before the earliest cutoff you passed that never had one (a publish whose delivery was purged in the same instant, or an event with no subscriber). Configure it through the bus `retention` option (`retention.completed` is therefore the horizon of event deduplication and of the events' storage). Delete rows through `purge` only; events are removed by cascade or by purge, never by hand while publishers are active. - **Deduplication horizon.** An event remembers its handlers only as long as the event row exists. Once `purge` deletes the event (older than `retention.completed` and without deliveries), a broker redelivery or publish retry with the same id is a new event and runs its handlers again; keep `retention.completed` longer than any redelivery window. - **Lock duration.** A publisher holds its key's counter row until commit. Keep transactions that publish keyed events short; unkeyed events take no such lock. - **Key counters.** `key_sequences` keeps one row per key ever published and rows are never deleted; the table grows with the number of distinct keys (a bigint counter each). Pick keys with bounded cardinality, or prune counters of keys you know are retired once nothing for them is pending. - **Stored JSON.** Payloads and headers are stored as `json`, the text the producer serialized: an event read back serializes byte for byte as it was published, key order included. (Before 0.4.1 they were `jsonb`, which re-orders object keys; `setup()` converts the columns, and rows stored before the upgrade keep the order `jsonb` gave them. See [Upgrading](https://berkcansavur.github.io/relaybus/upgrading).) A string containing `\u0000` (NUL) or a lone UTF-16 surrogate is rejected by PostgreSQL and fails the publish with a database error; strip or encode such text before publishing. - **Sequence gaps.** Republishing an existing event id consumes a sequence number. Gaps are normal; only order matters. - **Vacuum.** Completing a delivery deletes its row, which leaves a dead tuple until autovacuum runs. The tables are created with `autovacuum_vacuum_scale_factor = 0.02`; for sustained throughput above a few thousand deliveries per second also lower `autovacuum_vacuum_cost_delay` for this table (or run a periodic `VACUUM`), and watch `pg_stat_user_tables.n_dead_tup`. The lane table is vacuumed by HOT pruning alone. - **Lost answers and stale reads.** A claim is one statement. If the connection fails after the database recorded the batch but before the answer arrived, the store gives back every delivery of its own lanes that it never handed out (it remembers what it handed out until the dispatcher settled it). A claim that waited for a lane lock re-runs itself when the lane changed meanwhile, so it never claims from a snapshot older than the lane it got. The dispatcher retries a completion, reschedule, bury or release that the store did not answer until it lands or the lease is gone, so a delivery cannot stay in flight under a lane that other deliveries keep renewing. - **Lane exclusivity.** While a worker holds a lane under a live lease, other workers do not see its pending deliveries, even when they sit idle. A worker that dies delays its lanes by at most the lease duration (`dispatcher.leaseDuration`); a worker that stops gracefully gives them back immediately. Lanes are chosen per claim, so a busy backlog spreads over workers; a few hot keys live in few lanes and run on few workers by design (they are ordered). - **Round trips.** A claim is normally one statement: it claims from the lanes the worker holds and takes free lanes in the same statement. Only taking over a lane whose previous holder died with deliveries running (their lost attempts are charged first) continues in a short transaction. Publishing new events is one statement per `append`, whatever the batch size; adding a handler to an event that already exists costs two more. - **Wake-ups.** With `wakeUp: 'notify'` (the default) a publish that inserted a delivery due now notifies the dispatchers (none for delayed deliveries; PostgreSQL merges repeats within a transaction and drops them on rollback). A store sends at most one notification per 10 ms from its publishing statements; publishes in between leave one trailing notification that the store sends after their commit, so concurrent publishers do not queue behind PostgreSQL's notification lock. A dispatcher on a store built from `pool` or `connectionString` listens on that channel (`.deliveries`, hashed when longer than 63 characters) on one connection of its own, opened with the pool's settings outside the pool, so an event published by any process runs within milliseconds instead of at the next poll. A lost listening connection is replaced after a second; polling remains the safety net, so a missed notification only costs latency. The notification of a publish in your own transaction is sent by PostgreSQL at its commit, however late that is, so the handler starts within milliseconds of the commit (`wake-after-commit.integration.test.ts`). A store built on an `executor` listens only when it is given `pool` as well; without it, it relies on polling and on the wake-ups of its own process after a publish (see `bus.wake()`). `wakeUp: 'poll'` sends and listens for nothing. - **Prepared statements.** With `preparedStatements: true` (the default) the publish, claim, completion and renewal statements are named prepared statements: each connection parses and plans them once. Only `PgExecutor` (the default `pool` / `connectionString` path, and the `pool` given next to an `executor`) prepares; the library adapters send the statements of your transactions unprepared (a dispatch batch on an ORM store parses 2 statements without `pool`, 0 with it: `perf-design-orm-executors.integration.test.ts`). See `docs/performance/publish-path.md` for what that saves. - **Pooling.** Claim, complete, extend and purge need a free connection besides your transactions; size the pool for `dispatcher.concurrency` plus publishers; the listening connection comes on top of the pool's `max`. Listening and prepared statements need a session: with PgBouncer in transaction pooling use `wakeUp: 'poll'`, and `preparedStatements: false` unless PgBouncer is 1.21 or newer with `max_prepared_statements` above 0. - **Handlers added later.** A delivery added to an old event (new handler, republish with the same id) keeps the event's position in its key; if later events of that handler already finished, it runs after them. - **Listing completed deliveries.** Without `completionHistory`, `listDeliveries({ status: 'completed' })` is empty and `statistics().completed` is 0 (rows left by 0.1.0 excepted). With it, every completion also inserts one row into `completed_deliveries` (one more heap and index write per delivery); use it for audits, not for throughput. ## Troubleshooting - `relation "relaybus_events" does not exist`: `setup()` did not run; start the bus with `setupStore: true` or apply `createPostgresSchemaSql()`. - `permission denied for schema`: the role needs `CREATE` on the schema for `setup()`; with `setupStore: false` it needs only DML rights. - `invalid input syntax for type json` / `unsupported Unicode escape sequence`: the payload holds a NUL character or a lone UTF-16 surrogate, which PostgreSQL rejects. - Foreign key error on append: the only known cause is republishing an event id in the same instant its last delivery is purged. The publishing transaction rolls back and can simply be retried. - A key looks stuck: a pending or processing head blocks it. Inspect `listDeliveries({ status: 'processing' })` and `requeue` dead heads; expired lane leases are taken over automatically. - Deliveries wait although workers are idle: another worker holds their lane under a live lease (look at `lanes.lease_owner` and `lease_expires_at`); it is released when that worker finishes, stops, or the lease expires. - `deadlock detected` (`40P01`) or `could not serialize access` (`40001`) while publishing: see Deadlocks and Isolation under Operations; retry the transaction. ## Testing Integration tests use Testcontainers against `postgres:17-alpine` and `postgres:13-alpine` (override with `RELAYBUS_POSTGRES_IMAGES`). Every adapter (Kysely, Knex, TypeORM, Drizzle, Prisma with and without `pool`) runs the full `@relaybus/testing` event store contract plus transaction tests against the real database. ## When to use @relaybus/postgres, and when not to Use it when your data lives in PostgreSQL 13 or newer and events must commit or roll back with it. It is not for MySQL or MongoDB (see `@relaybus/mongodb` for MongoDB). Documentation: ### relaybus vs alternatives | | What it is | Survives a restart | Written in your database transaction | Choose it when | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | **relaybus** | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries | Yes, events live in your database | Yes | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys | | [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2` | No persistence is documented | No | Simple in-process notifications where losing an event on restart is acceptable | | [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs) | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory | No built-in persistence | No | You want CQRS structure; it can sit on top of an outbox | | [`@nestjs/outbox`](https://github.com/nestjs/outbox) | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores | Yes | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing | | [BullMQ](https://docs.bullmq.io/) | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case | Yes, in Redis | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity | | [pg-boss](https://github.com/timgit/pg-boss) | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction | Yes, in PostgreSQL | Yes, through its database adapters | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports | | Kafka client directly | A distributed log; the idempotent producer and transactions give exactly-once within Kafka | Yes, in Kafka | No, producing to Kafka is a second write | You consume and produce only inside Kafka; add an outbox when a database write must be announced | This table summarises the vendors' own documentation as of October 2026; check their docs for current details. Full comparison and sources: . --- Source: https://berkcansavur.github.io/relaybus/packages/mongodb # @relaybus/mongodb @relaybus/mongodb is the MongoDB event store of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: it writes the transactional outbox in your own multi-document transaction, with the MongoDB driver or Mongoose sessions. MongoDB event store for [relaybus](https://github.com/berkcansavur/relaybus): a transactional outbox on multi-document transactions, with lane leases, batched claims and completions, and per-key ordering. Events are written in the **same MongoDB transaction** as your business data, so a committed change always has its event and a rolled-back change never leaks one. Delivery is at-least-once; handlers registered with `transactional: true` also get effectively-once processing, because their writes and the delivery's completion commit together. ## Requirements - MongoDB 6.0 or newer, **as a replica set or a sharded cluster**. MongoDB only supports transactions there. A single-node replica set is enough for development. - Node.js 22.12 or newer, `mongodb` driver 6 or 7 (peer dependency), `@relaybus/core`. A standalone `mongod` is rejected with a `ConfigurationError` that says so (from `setup()` and from the first transaction). Start a development replica set with `mongod --replSet rs0` followed by `rs.initiate()`, or use the `mongo` Docker image the same way. ## Install ```sh npm install @relaybus/core @relaybus/mongodb mongodb ``` ## Usage ```ts import { MongoClient } from 'mongodb'; import { RelayBus, defineEvent } from '@relaybus/core'; import { MongoEventStore } from '@relaybus/mongodb'; import type { ClientSession } from 'mongodb'; class OrderPlaced extends defineEvent<{ orderId: string }>('order.placed.v1') {} const client = await new MongoClient(process.env.MONGODB_URI!).connect(); const bus = new RelayBus({ store: new MongoEventStore({ client, database: 'shop' }), }); bus.subscribe(OrderPlaced, async (event) => sendConfirmation(event.payload.orderId), { id: 'send-confirmation', }); await bus.start(); // creates collections and indexes, then starts dispatching await bus.transaction(async (session) => { await client.db('shop').collection('orders').insertOne({ _id: 'o-1' }, { session }); await bus.publish(new OrderPlaced({ orderId: 'o-1' }), { key: 'o-1' }); }); ``` The transaction handle is the driver's `ClientSession`. Pass it to every write that must be atomic with the event. ### Using an existing session or Mongoose `publish({ transaction })` and `store.append(session, ...)` accept any `ClientSession` that is inside a started transaction, so you are not forced to use `bus.transaction`. A session without a started transaction is rejected with a `ConfigurationError`: MongoDB would commit each write immediately, silently breaking the atomicity the outbox exists for. Mongoose exposes the same driver session (`mongoose.startSession()`, `Model.create([doc], { session })`), and `mongoose.connection.getClient()` returns the `MongoClient` to give to the store: ```ts const session = await mongoose.startSession(); await session.withTransaction(async () => { await Order.create([{ total: 10 }], { session }); await bus.publish(new OrderPlaced({ orderId: '1' }), { transaction: session }); }); ``` relaybus does not depend on Mongoose. `session.withTransaction` may run your callback again after a transient error (see [Transactions and retries](#transactions-and-retries)). ### Transactional handlers ```ts bus.subscribe( OrderPlaced, async (event, context) => { await projections.insertOne( { orderId: event.payload.orderId }, { session: context.transaction }, ); }, { id: 'projector', transactional: true }, ); ``` The projection write and the delivery's completion commit together or not at all. ## Options | Option | Default | Meaning | | -------------------- | ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | | `client` | required | A connected `MongoClient`. The store never closes it. | | `database` | the client's default database | Database for the collections. 1-63 characters of letters, digits, `_`, `-`. | | `collectionPrefix` | `'relaybus_'` | Collections are `events`, `deliveries`, `key_sequences`, `lanes`. | | `completionHistory` | `false` | Keep an append-only record of completed deliveries (`completed_deliveries`). Same value in every process. | | `transactionOptions` | snapshot read concern, `w: 'majority'`, primary | Passed to `withTransaction`. | | `purgeBatchSize` | `500` | Documents deleted per purge step. | | `logger` | `console` | Receives warnings about best-effort steps that failed (parking). | | `generateIdentifier` | - | Deprecated and ignored: a lease token is `::`. | Non-transactional writes (claims, renewals) use write concern `majority` and primary reads, whatever the client default is, so a lease survives a failover and the ordering check never reads a stale secondary. ## How it works **Collections.** `events` (`_id` = event id), `deliveries` (one per unfinished event and handler), `key_sequences` (one counter document per ordering key), `lanes` (256 lane documents), and `completed_deliveries` with `completionHistory`. **Idempotent append.** Event ids are deduplicated by reading what exists inside the transaction and writing with upserts. A plain insert would raise a duplicate-key error (11000) when two transactions race on the same id, and a duplicate-key error aborts the whole transaction without a retry label. An upsert race surfaces as a retryable write conflict instead. Every event lists the handlers it ever had a delivery for (`handlers`), completed ones included: a redelivered broker message or a publish retry adds deliveries only for handlers the event does not list yet, so a handler that already completed never runs twice although its delivery was deleted. The unique index on `(eventId, handler)` is a backstop; the code never relies on catching it. **Per-key sequence equals commit order.** Appending a keyed event runs `findOneAndUpdate({ _id: key }, { $inc: { value: 1 } }, { upsert: true })` on the counter inside the append transaction. A transaction holding an uncommitted increment makes every concurrent appender of that key hit a write conflict; the loser restarts after the winner committed and reads the next value. Two transactions can never both hold sequence `n`, and a lower sequence is always committed before a higher one, so ordering by sequence is ordering by commit. **Lane leases.** Every delivery belongs to one of 256 lanes, `laneId`: the 32-bit FNV-1a hash of the UTF-8 ordering key, or of the event id when there is none, modulo 256. All deliveries of a key share a lane. Workers lease lanes, not deliveries. A lane document holds the owner, the lease expiry, an `epoch` and `inFlight`, the ids of the deliveries the owner claimed. A delivery runs while its document exists as `pending` and its lane lists it; `processing` is derived, never stored. A claim: 1. claims from the lanes the worker already holds under a live lease; 2. if it still needs more, takes free lanes (no owner, or an expired lease) that have due deliveries of its handlers: 4, then 8, 16, ... lanes per round, the lane with the oldest due delivery first, so a worker holds only as many lanes as its batch needs. Taking a lane is one conditional `findOneAndUpdate` that increments the epoch; a lane another claimer took first is skipped, never waited for; 3. reads the due, ready deliveries of those lanes in `(dueAt, _id)` order, leaving out the ones it already runs, applies the per-key rules below, and stops at `limit`; 4. records the batch with one conditional write per lane, all in one `bulkWrite` (`inFlight` gains the new ids and loses the ids whose documents are gone; the lease never shrinks), and hands back the lanes that yielded nothing and run nothing. A claim of an unkeyed batch therefore costs a small, fixed number of round trips and writes **no delivery document**. A completion deletes the delivery documents and writes no lane document. Delivery documents are inserted once and deleted once, and no indexed field of theirs changes on the normal path. Lanes are held per store instance as well as per `leaseOwner`: a process restarted with a fixed owner name does not adopt the lanes of its dead predecessor, it takes them over once their leases expired, like any other worker. A claim reads the events of its batch before recording it, and a pick that a dead letter requeued during the claim now precedes is given back right after recording, without an attempt charged; parking is best effort and never fails a claim (failures go to the `logger` option, default `console`). **Lane exclusivity.** While a worker holds a lane under a live lease, other workers do not see the pending deliveries of that lane, even idle ones. A worker that dies delays its lanes by at most the lease duration; a worker that stops gracefully hands them back at once (`relinquish`). Lanes are chosen per claim, so a backlog spreads over workers; a few hot keys live in few lanes and therefore run on few workers at a time. **Per-key order.** A keyed delivery is claimed only when no earlier delivery (lower sequence) of its handler and key is pending (a running one is still a pending document) and no other delivery of them runs. Because all deliveries of a key share one lane and only the lane owner claims from it, the running ones are exactly the ones listed by the owner's lanes, which the claim loads once. No delivery is locked or written to check this. One claim call never takes two deliveries of the same key, and what it learns about a key answers every later delivery of that key without another query. The scan continues past blocked deliveries until `limit` is reached, so a hot blocked key cannot starve unrelated work. **Parked backlogs.** A key whose head is slow can accumulate thousands of deliveries that are all due and all blocked. Scanning them on every claim would cost time proportional to the backlog. When one claim finds 32 or more deliveries of a key waiting behind an unfinished predecessor, it moves them to the `parked` claim lane (the `lane` field), which the claim index leaves out. A claim that hands a lane back also parks the waiting deliveries it found there, however few: they are all the due work of that lane (typically followers of a head whose retry is not due yet), and leaving them ready would make every later claim take the lane for nothing. At the start of every claim, the lowest parked delivery of each key with no unfinished predecessor is moved back to `ready` (one aggregation per handler, a distinct scan of the small parked index, then one predecessor lookup per parked key, in parallel, and one update). This is deliberately level-triggered: it looks at the current state, never at an event, so a crash while parking, or a predecessor finishing while its followers were being parked, can delay a key until the next claim but never strand it. Parked deliveries keep their `dueAt`, so `statistics().oldestDueAt` still counts them. The per-claim cost of the move-back step is one index probe and one parallel lookup per key that currently has parked deliveries, so keep the number of such simultaneously backlogged keys in the hundreds, not millions. Parking and unparking write delivery documents; they are the slow path for blocked keys only. `requeue` is an operator action and makes a dead delivery pending again even if later deliveries of its key already ran; it blocks the ones that have not been claimed yet, and it cannot start while a later delivery of its key is running. **Fencing.** A lease token is `::`. `claims` is the delivery's claim generation: it grows whenever a claim of the delivery ends (retry, release, burial, lost lease), so the lease of an earlier claim never matches a later claim of the same delivery, even by the same owner in the same lane epoch. `complete`, `completeMany`, `reschedule`, `release`, `bury` and `extendLease` apply only while the lane still has that epoch and lists the delivery, and the delivery still has that generation (a missing generation, as in documents of 0.1.0, counts as 0); a worker that lost its lane changes nothing. Claims of one owner on one store instance run one after the other, so overlapping claims (`bus.runOnce()` next to a running dispatcher) never read the same in-flight list. Renewals check the claim generation and extend the lane in one transaction, so a stale lease never extends the lane of a newer claim. A claim that fails at any point, even after its record reached the database without the reply reaching the process, reconciles the owner's lanes on this instance in one transaction: a listed delivery stays listed only if this instance handed out its current claim (current lane epoch and claim generation, exact lease token) and has not seen it settled; every other one is taken out and its claim generation moves on, so no old lease can match a later claim of it. A worker taking such a lane over meanwhile conflicts with the reconciliation, which then leaves its claims alone. A token this store cannot have issued is refused without a database call. An expired lease nobody took over is still honoured. **Completion and takeover.** A worker that takes a lane over first writes every delivery the previous owner still had in flight (it charges the lost attempt, see below) and only then reads what to claim. A completion never writes a lane document, so concurrent completions of one lane never conflict with each other; it relies on that charge instead: - `complete(lease, now, session)` (transactional handlers) and every completion when `completionHistory` is on run in a transaction that reads the lanes and deletes the deliveries (and inserts the history records). The completion and a takeover write the same delivery document, so whichever commits second either hits a write conflict and is retried against the new state (seeing the new epoch, or the delivery gone), or waits for the first to finish. - Otherwise `complete` and `completeMany` need no transaction: they read the deliveries' claim generations, then the lanes, and delete only the deliveries that still have the lease's generation. A takeover before the lane read shows up in it as a new epoch; a takeover, retry or release after it changed the generation, so the delete matches nothing. A batch of any size from any number of lanes costs four commands (the fourth records the completion time on the events). When fewer deliveries were deleted than leases were fenced (a claim ended or changed hands meanwhile), the call reports no lease as completed: a missing document cannot be told apart from one the new claim completed, so success is reported only when it is proven. A completion and a takeover can never both succeed, which is what makes a transactional handler's writes effectively-once. Two concurrent completions of the same lease without a transaction may both report success; the dispatcher never issues them. **Attempts.** `attempts` in a delivery document counts attempts that **ended**: `reschedule` and `bury` add the failed one, `release` (graceful shutdown) adds nothing, and the claim result reports `attempts + 1`. When a lane is taken over, every delivery the previous owner still had in flight is charged one attempt, so a worker that keeps crashing on a poison message still runs it into the retry limit and the dead letters. The take moves the previous owner's in-flight list to the lane's `lost` list in the same write; `lost` is charged before anything is read from the lane and cleared afterwards, and whoever holds the lane next charges a `lost` list it finds first. A claim that fails anywhere after the take therefore loses no charge and leaves nothing listed as running. Charging and clearing are separate writes, not a transaction, but each charge is recorded on the delivery (`lostAt`: the lane epoch of the last lost claim charged, which only grows), so charging the same list again after a failure between them, or a delayed charge of an older lost claim, charges nothing twice. The charge also moves the claim generation on. **Completion history.** Completed deliveries are deleted. With `completionHistory: true` every completion also inserts a record into `completed_deliveries` in the same transaction. Records are listed by `listDeliveries` as `completed` (all filters apply), counted by `statistics().completed` and removed by `purge({ completedBefore })`. Without it, `listDeliveries({ status: 'completed' })` is empty and `statistics().completed` is 0 (except for `completed` documents left by 0.1.0). **Time.** The store uses only the `now` it is given, never the server clock. Lease expiry is compared with the `now` of the claimer, so keep worker clocks synchronized. **Payloads.** Payload and headers are stored as the JSON-normalized value: `Date` becomes an ISO string, `undefined` members disappear, as in every other relaybus store. Keys containing `.` or `$` are stored as they are (MongoDB 5+). Text with a **lone UTF-16 surrogate** (a broken emoji, for example) is rejected with an `EventValidationError`: BSON strings are UTF-8, so MongoDB would silently store U+FFFD instead. Run such text through `String.prototype.toWellFormed()` first. **Untrusted input.** The fields of `listDeliveries` and the ids of `requeue` are coerced to strings before they reach a query, so an object parsed from an admin request cannot become a query operator such as `{ $ne: ... }`. ## Transactions and retries `store.transaction(work)` uses `session.withTransaction`, which restarts the **whole callback** after a `TransientTransactionError` (typically a write conflict) and re-sends only the commit after an `UnknownTransactionCommitResult`. For everything written through the session this is exactly what you want: an aborted attempt leaves nothing behind, and `append` re-reads state at the start of every attempt, so a retry never appends twice. The value returned is that of the final attempt. The consequence is that `work` can run more than once, and the same holds for `bus.transaction`, which is built on it. Keep side effects that do not go through the session (HTTP calls, other databases, message brokers) out of it, or make them idempotent. This fits relaybus: handlers are at-least-once anyway, and event publishing is the mechanism meant to replace such side effects. Retries stop after 120 seconds (the driver's limit) with the last error. Transactional handlers run inside such a transaction too, so the same rule applies to them. The core does not renew the lease of a transactional handler while it runs (its completion must not conflict with a renewal), so the lease must outlast the handler: keep transactional handlers short, as the core documentation recommends. ### Transaction time budget `bus.transaction` passes the bus `transactionTimeout` (default `'5s'`) to the store, which bounds the commit on the server with `maxCommitTimeMS`. A transaction that ran past the budget is aborted before its commit, with all its writes, and the caller gets a `TransactionTimeoutError`. The driver's `timeoutMS` is not set on the transaction, because it would make operations that pass their own `timeoutMS` with the session throw. The store's own transactions are not affected. ## Indexes Created idempotently by `setup()` (also run by `bus.start()` unless `setupStore: false`): | Collection | Index | Used by | | ---------------------- | ----------------------------------------------------- | ---------------------------------------------- | | `deliveries` | `(eventId, handler)` unique | idempotent append, lookup by event | | `deliveries` | `(laneId, dueAt, _id, handler)` partial | claim, free-lane discovery (`ready` lane only) | | `deliveries` | `(handler, key, sequence)` partial | predecessor check of keyed deliveries | | `deliveries` | `(handler, key, sequence)` partial | lowest parked delivery of each key (`parked`) | | `deliveries` | `(dueAt)` partial | `statistics().oldestDueAt` | | `deliveries` | `(completedAt)` partial | purge of `completed` documents left by 0.1.0 | | `deliveries` | `(status, createdAt)` | status counts, listing by status | | `deliveries` | `(status, updatedAt)` | purge of dead deliveries | | `deliveries` | `(handler, createdAt)` | listing by handler | | `deliveries` | `(eventType, createdAt)` | listing by event type | | `events` | `(createdAt, _id)` | purge of events nothing references | | `completed_deliveries` | `(completedAt, _id)`, `(eventId)`, `(createdAt, _id)` | purge, event references, listing | The claim index ends with `handler` so that a query over many lanes stays a merge of one index scan per lane (the planner merges at most 200), and a claim queries at most 128 lanes at once. The lane collection has 256 small documents and no secondary index. `setup()` also creates the collections explicitly, because collections created implicitly inside concurrent transactions conflict with each other, and seeds the lane documents. If you manage the schema yourself, run `setup()` once from a deployment script instead. Partial indexes contain only unfinished (or legacy completed) deliveries, so they stay small. If you never use `listDeliveries` filters, the three listing indexes can be dropped at the cost of slower operator queries. ## Upgrading from 0.1.0 The step-by-step guide, including `completionHistory` and PgBouncer settings, is [Upgrading from 0.1 to 0.2](https://berkcansavur.github.io/relaybus/upgrading). `setup()` upgrades a 0.1.0 database in place, idempotently, and is safe to run again: `processing` deliveries become `pending` and due at once (their attempt stays counted, as a lost lease would count it), the per-delivery lease fields are removed, every delivery gets its `laneId` and every event its `handlers` and `createdAt` (from its deliveries, in batches of 1,000), the `running` index and the 0.1.0 `claim` index are dropped, and the lane documents are created. The lane documents mark the upgrade as done, so later runs skip it. `completed` documents left by 0.1.0 stay until a purge with `completedBefore` removes them. Stop all 0.1.0 workers before the first `setup()`: they cannot work with the upgraded documents. ## Tuning - **Batch size.** A claim takes 4, 8, 16, ... lanes until it has `limit` deliveries, so a worker's lane count follows its batch size. Unkeyed deliveries spread over all 256 lanes; very small batches with many workers keep lanes changing hands often, very large ones let one worker hold most lanes. - **Lease duration.** A lane whose owner died is unavailable for at most the lease duration. Keep it well above the handler run time (the dispatcher renews it while handlers run). - **Completion batching.** The dispatcher completes deliveries that finish close together with one `completeMany`: three commands per batch (one transaction with `completionHistory`). - **Transactional handlers** commit their writes with the deletion of the delivery. They conflict only with a takeover of their own delivery, not with each other. - **Document size.** New events leave out `key`, `sequence`, `correlationId`, `causationId` when they are null and `revision` while it is 0, and new deliveries leave out `updatedAt` until it differs from `createdAt`: every byte of a new document is written to the oplog too. Documents of earlier versions with these fields are read the same way. ## Retention `bus.purgeExpired()` (run on a schedule by the bus) calls `store.purge`, which deletes, in batches of `purgeBatchSize`, dead deliveries (`deadBefore`), completion history records and legacy `completed` documents (`completedBefore`), each batch together with the events nothing references any more **in one transaction**, so a crash can never leave a delivery without its event. Then it deletes events nothing references. Every completion records its time on the event (`completedAt`, the latest one), and such an event is kept as deduplication record until `completedBefore` passes that time, however old the event is; `deadBefore` alone never removes it. Events that never had a completion (no subscribers, or only dead letters) go once they were created before the earliest given cutoff. An append that adds a delivery to an event while it is being purged writes the event, conflicts with the purge, and one of them restarts, so a delivery never loses its event. An event deleted by purge takes its `handlers` list with it: a broker redelivery or publish retry of that event id after the purge is appended as a new event and runs its handlers again (the same holds for the PostgreSQL store). Keep completed retention longer than any redelivery window. One counter document per distinct ordering key stays in `key_sequences`. They are tiny; if your keys are unbounded (for example one per request), clean up counters of long-finished keys yourself. ## Troubleshooting - **`MongoEventStore needs multi-document transactions`**: the server is a standalone `mongod`. Use a replica set or sharded cluster. - **`Transaction numbers are only allowed on a replica set member or mongos`** raised by your own code: same cause, for your own transactions. - **`TransientTransactionError` / write conflicts under load**: expected when many publishers append the same key at once; they serialize through the key's counter and retry. Spread hot keys or batch publishes with `publishAll`. - **Deliveries stay pending**: check `statistics().oldestDueAt`, and that the process that publishes also registered the handler (deliveries are created from the publisher's registrations). - **Deliveries wait although workers are idle**: another worker holds their lane under a live lease (look at `lanes`: `owner`, `leaseExpiresAt`); it is released when that worker finishes, stops, or the lease expires. - **A key is stuck**: a pending, delayed, running or requeued delivery earlier in the same key blocks later ones, and so does any running delivery of the key; inspect it with `listDeliveries({ handler })` and `bus.requeue` or bury it. - **Slow claims with a very large backlog of one key**: backlogs of 32 or more waiting deliveries are parked automatically; check that the `claim`, `ordering` and `parked` indexes exist (`setup()`). Deliveries with `lane: 'parked'` are expected while their key is blocked. - **`MongoEventStore.append needs a ClientSession inside a started transaction`**: the session passed to `publish({ transaction })` or `append` has no transaction. Start one with `session.withTransaction` or `session.startTransaction()`. ## When to use @relaybus/mongodb, and when not to Use it when your data lives in MongoDB 6 or newer running as a replica set or sharded cluster. MongoDB transactions need a replica set: a standalone `mongod` is rejected. Documentation: ### relaybus vs alternatives | | What it is | Survives a restart | Written in your database transaction | Choose it when | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | **relaybus** | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries | Yes, events live in your database | Yes | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys | | [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2` | No persistence is documented | No | Simple in-process notifications where losing an event on restart is acceptable | | [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs) | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory | No built-in persistence | No | You want CQRS structure; it can sit on top of an outbox | | [`@nestjs/outbox`](https://github.com/nestjs/outbox) | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores | Yes | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing | | [BullMQ](https://docs.bullmq.io/) | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case | Yes, in Redis | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity | | [pg-boss](https://github.com/timgit/pg-boss) | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction | Yes, in PostgreSQL | Yes, through its database adapters | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports | | Kafka client directly | A distributed log; the idempotent producer and transactions give exactly-once within Kafka | Yes, in Kafka | No, producing to Kafka is a second write | You consume and produce only inside Kafka; add an outbox when a database write must be announced | This table summarises the vendors' own documentation as of October 2026; check their docs for current details. Full comparison and sources: . --- Source: https://berkcansavur.github.io/relaybus/packages/kafka # @relaybus/kafka @relaybus/kafka is the Apache Kafka transport of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: it forwards committed outbox events to Kafka topics and stores consumed events once in an idempotent inbox, so events are not lost between your database and Kafka. Apache Kafka transport for [relaybus](https://github.com/berkcansavur/relaybus#readme). It forwards committed outbox events to Kafka topics and feeds events consumed from Kafka into the bus inbox, with at-least-once delivery and idempotent storage. - Wire format: [CloudEvents Kafka protocol binding](https://github.com/cloudevents/spec/blob/main/cloudevents/bindings/kafka-protocol-binding.md), binary content mode, so any CloudEvents-aware consumer or tool can read the topic. - Tested on a real broker with both clients: [`@confluentinc/kafka-javascript`](https://github.com/confluentinc/confluent-kafka-javascript) 1.11.0 (its KafkaJS-compatible API, **recommended**: maintained, librdkafka based) and [`kafkajs`](https://kafka.js.org) 2.2.4 (unmaintained). See [Choosing a client](#choosing-a-client). Neither client is a dependency; install the one you use. - Never loses an event: the offset is committed only after the bus stored the event; a failing store blocks the partition instead of skipping. ## Install ```sh pnpm add @relaybus/core @relaybus/kafka @confluentinc/kafka-javascript ``` ## Usage ```ts import { KafkaJS } from '@confluentinc/kafka-javascript'; import { RelayBus } from '@relaybus/core'; import { KafkaTransport } from '@relaybus/kafka'; const kafka = new KafkaJS.Kafka({ kafkaJS: { clientId: 'orders', brokers: ['kafka:9092'] } }); const transport = new KafkaTransport({ kafka, client: 'confluent', // the default is 'kafkajs' groupId: 'billing', // required to subscribe }); const bus = new RelayBus({ store, source: 'orders', transports: [transport], routes: [{ event: OrderPlaced, transport: 'kafka', destination: 'orders.events' }], subscriptions: [{ transport: 'kafka', destinations: ['payments.events'] }], }); await bus.start(); // throws if the consumer cannot join its group ``` Readiness for Nest or Kubernetes: ```ts // true only when the producer is connected and, if subscribed, the consumer is consuming const ready = transport.isReady(); // used by the bus health check // false while rebalancing, after a crash, after close, or while any partition is stuck failing const consuming = transport.isConsuming(); ``` ## Options | Option | Default | Meaning | | ------------------------------ | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `kafka` | required | A `kafkajs` `Kafka` or Confluent `KafkaJS.Kafka` instance: the factory of the transport's consumer and producer. Optional with `producer`, but then the transport cannot subscribe. | | `producer` | none | An already-connected producer the application owns (kafkajs `Producer`, Confluent `KafkaJS.Producer`), used to publish and dead-letter instead of one of `kafka`. Never connected or disconnected by the transport. See [Reusing the application's producer](#reusing-the-applications-producer). | | `client` | `'kafkajs'` | Which client `kafka` is: `'kafkajs'` or `'confluent'`. Set `'confluent'` for a Confluent `KafkaJS.Kafka`; see [Choosing a client](#choosing-a-client). | | `name` | `'kafka'` | Transport name used by routes and subscriptions. | | `groupId` | none | Consumer group. Required as soon as `subscribe` is used; publish-only services can omit it. | | `fromBeginning` | `true` | Where a group without committed offsets starts. `true` so events published before the first start are not skipped. | | `maxInFlightRequests` | `1` | Producer in-flight requests, 1 to 5. `1` keeps retries from reordering events of one key. | | `producerConfig` | `{}` | Extra producer settings (partitioner, compression, ...). `idempotent` and `maxInFlightRequests` always win. Confluent: KafkaJS-style settings in a `kafkaJS` block, librdkafka properties at the top level; `acks: -1` also wins, as do `enable.idempotence`, `max.in.flight` and `acks` there. | | `consumerConfig` | `{}` | Extra consumer settings (`sessionTimeout`, `retry`, `heartbeatInterval`, ...). `groupId` always wins. Confluent: KafkaJS-style settings in a `kafkaJS` block, librdkafka properties at the top level; `fromBeginning` also wins, as do `group.id` and `auto.offset.reset` there. Your `rebalance_cb` is still called. | | `processing.maxAttempts` | `5` | In-process attempts per delivery of a record. | | `processing.initialBackoff` | `'200ms'` | Delay before attempt 2, doubling per attempt. | | `processing.maxBackoff` | `'5s'` | Upper bound of the delay between attempts. | | `processing.heartbeatInterval` | `'1s'` | Heartbeat interval while sleeping between attempts, so a long backoff never exceeds `sessionTimeout`. | | `startupTimeout` | `'30s'` | How long `subscribe` waits for the group join. A failed start-up disconnects the consumer it started. | | `publishTimeout` | `'30s'` | How long `publish` waits for the broker ack before rejecting with `PublishTimeoutError`. | | `restart.initialBackoff` | `'1s'` | First delay of the consumer restart loop after an unrecoverable crash; doubles per failure. | | `restart.maxBackoff` | `'30s'` | Upper bound of the restart delay. | | `onInvalidMessage` | `'dead-letter'` | `'dead-letter'` (to `.dead`), `{ deadLetterTopic: string \| (sourceTopic) => string }`, a callback `(message) => void \| Promise`, or `'skip'` (drops the record; explicit opt-in). | | `codec` | `cloudEventsBinary` | The wire format, both ways: `cloudEventsBinary` (CloudEvents binary mode, strict on read), `cloudEventsStructured`, `jsonEnvelope({...})` (your JSON envelope, or the bare payload) or `{ encode, decode }`. See [Interoperating with services that do not use relaybus](#interoperating-with-services-that-do-not-use-relaybus). | | `decode` | none | Deprecated: reads with your function, writes binary mode. Same as `codec: { encode: encodeEnvelope, decode }`; not together with `codec`. | | `logger` | `console` | `log`, `warn`, `error`. Messages start with `[KafkaTransport](method)`. | ## Wire format Value: the JSON payload. Key: the event key (`null` means no key); `jsonEnvelope` can write another key or none (`recordKey`). | Header | Content | | ------------------------------------ | --------------------------------------------------------- | | `ce_specversion` | `1.0` | | `ce_id`, `ce_type`, `ce_source` | envelope `id`, `type`, `source` | | `ce_time` | `occurredAt` as RFC 3339 UTC (`2026-01-01T00:00:00.000Z`) | | `ce_subject` | envelope key, only when present | | `content-type` | `application/json` | | `ce_correlationid`, `ce_causationid` | relaybus extensions, only when present | | anything else | envelope headers, passed through | An envelope header whose name collides with the protocol (`ce_*`, `content-type`, `x-relaybus-*`, any case) is stored as `x-relaybus-h-` and restored on read, so user headers can never overwrite event attributes and round-trip unchanged. Decoding is strict: wrong spec version, missing or empty `ce_id`/`ce_type`/`ce_source`, an invalid event type, a non-RFC 3339 `ce_time`, a non-JSON content type, a missing or non-JSON/UTF-8 value, repeated protocol headers, or a `ce_subject` that disagrees with the record key all make the record malformed. `ce_time` is optional in CloudEvents: when a producer omits it, the Kafka record timestamp becomes `occurredAt`. relaybus itself always writes it. Payloads are JSON. An `undefined`, function, symbol, `BigInt` or circular payload, and an `occurredAt` that is an invalid date or lies outside the years 0 to 9999 (not representable as RFC 3339), are rejected on publish with `UnencodableEventError`, a `NonRetryableError`, because retrying cannot help. `Date` payload values become ISO strings. A header named `__proto__` is kept as ordinary data by the codec (kafkajs itself drops it when it builds the consumer's header object). ## Interoperating with services that do not use relaybus **relaybus producer, other consumer.** Any Kafka client reads the record as is: the value is the JSON payload (UTF-8), the key is the event key, and the `ce_*` headers above carry the attributes. A plain consumer only needs `JSON.parse(value)` and the `ce_type` / `ce_id` headers. CloudEvents SDKs decode the record natively, because it is CloudEvents binary mode of the Kafka protocol binding: Java `io.cloudevents:cloudevents-kafka` (`CloudEventDeserializer`), Go `sdk-go` (`protocol/kafka_sarama`), and Spring Cloud Function / Spring Cloud Stream, which read the `ce_`-prefixed headers. Use `ce_id` to deduplicate: an outbox delivers at least once. **Other formats, both ways.** By default the transport reads only binary-mode CloudEvents; anything else is malformed and dead-lettered, so a misconfigured producer cannot slip events in. The `codec` option sets another wire format for publishing and consuming (one transport is one subscription): ```ts import { KafkaTransport, cloudEventsStructured, encodeEnvelope, fromHeader, jsonEnvelope, plainJsonDecoder, type KafkaCodec, } from '@relaybus/kafka'; // CloudEvents structured mode (`content-type: application/cloudevents+json`); reads binary too. new KafkaTransport({ kafka, groupId: 'billing', codec: cloudEventsStructured }); // A service envelope: `{ "eventName": ..., "messageId": ..., "data": {...}, "meta": {...} }`. const envelope = jsonEnvelope({ type: { body: 'eventName' }, id: { body: 'messageId' }, payload: { body: 'data' }, correlationId: { body: 'meta.correlationId' }, occurredAt: { body: 'meta.occurredAt' }, headers: { body: 'meta.headers' }, }); // The bare JSON payload of a Spring / Go / plain kafkajs producer, the type in a header. const plain = jsonEnvelope({ type: { header: 'event-type' }, id: 'record-position', // kafka:::, or { header: 'message-id' } payload: 'whole-body', }); // Your own: binary mode out, plain JSON and CloudEvents in. const custom: KafkaCodec = { encode: encodeEnvelope, decode: plainJsonDecoder({ type: fromHeader('event-type') }), }; ``` | Codec | Writes | Reads | | ----------------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------ | | `cloudEventsBinary` | Binary mode, as above. Default, byte for byte unchanged. | Binary mode only, strictly. | | `cloudEventsStructured` | One `application/cloudevents+json` document. | Structured and binary mode (`decodeCloudEvent`). | | `jsonEnvelope(mapping)` | The mapped JSON body and headers. | The same mapping. A missing type, mapped id or payload is malformed. | | `{ encode, decode }` | Whatever `encode` returns. | Whatever `decode` returns; missing fields are filled as for a decoder (table below). | `jsonEnvelope` mappings name a body path (`{ body: 'meta.correlationId' }`, `'messageType.0'` indexes an array) or a header (`{ header: 'event-type' }`) for `type`, `id`, `payload` (or `'whole-body'`), `key`, `source`, `occurredAt`, `correlationId`, `causationId` and `headers`, plus a `contentType`. `type`, `id` and `payload` are required by the types; `id: 'record-position'` uses the record position (records relaybus writes carry the event id in a `relaybus-event-id` header, which wins on read, so outbox retries stay deduplicated); `{ body: 'id', whenMissing: 'record-position' }` uses the body id when a record has one. The record key is the event key unless `recordKey` says otherwise (`'none'` or `(event) => string | null`), `contentTypeHeader: false` writes no `content-type` header, and when `occurredAt` is not mapped the record timestamp carries it. Event headers that are record headers keep their names, so a name the mapping uses, `content-type` or `relaybus-event-id`, is refused with `UnencodableEventError`. The [interoperability recipe](https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services) has a full table and a MassTransit example. Decoded payloads go through the event's schema (`defineEvent(type, { schema })`) as relaybus's own do; a payload that fails it makes the delivery dead with `EventValidationError`. The decoders below remain available for a custom codec's `decode`, and for the deprecated `decode` option: | Decoder | Accepts | | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | default (`decodeEnvelope`) | Binary-mode CloudEvents only, strictly as described above. Unchanged. | | `decodeCloudEvent` | Binary mode as the default, plus structured mode: a JSON value with `specversion` `1.0`, `id`, `type`, `source`, optional `time` (else the record timestamp), `subject` (else the record key), `data` (the payload; `data_base64` is refused). | | `plainJsonDecoder(opts)` | Any JSON value. A record that is a CloudEvent anyway (a `ce_specversion` header or a structured content type) goes through `decodeCloudEvent`, so relaybus and foreign producers can share a topic. | | your function | `(record) => ({ id, type, payload, ...optional })`. Missing fields are filled: `source` = topic, `key` = record key, `occurredAt` = record timestamp, `headers` = `{}`. | `plainJsonDecoder` options (each extractor gets `{ topic, partition, offset, key, headers, payload, timestamp }`; an extractor returning `undefined` or `''` selects the default): | Option | Default | Meaning | | ------------ | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------ | | `type` | required | A fixed event type, or an extractor such as `fromHeader('event-type')` or `fromPayloadField('type')`. | | `id` | `kafka:::` | Inbox dedup id. The default is stable across redeliveries of the record. Prefer a producer id when it may send an event twice. | | `key` | the record key | Ordering key. A key that is not UTF-8 (Java `LongSerializer`) is base64-encoded. | | `source` | the topic | Fixed text or an extractor. | | `occurredAt` | the record timestamp | Extractor returning a `Date`. | Every record header becomes an event header (the last value of a repeated header; a value that is not valid UTF-8 is left out, so binary ids can never collide). **Guarantees stay the same.** Decoding runs again on every redelivery and must be deterministic; the inbox deduplicates on the id, so a redelivered record is stored and handled once. Under every codec, a record that cannot be decoded (not JSON, a tombstone, no type, an invalid type, a missing mapped id, an extractor returning a non-string) and anything a custom `decode` throws go to `onInvalidMessage` (dead-letter by default) with the reason in `relaybus-invalid-reason`; the consumer keeps running. Per-key order is the Kafka partition order, as for relaybus producers. **Limits.** The derived id assumes offsets are never reused: deleting and recreating a topic restarts its offsets, so records after a recreation could collide with ids already in the inbox (within its retention). Two sends of one event by a producer are two offsets; only an id set by the producer (`id: fromHeader(...)` or `{ header }` / `{ body }` in a mapping) dedupes them. Avro or Protobuf need your own codec, whose functions must stay synchronous. ## Partitioning and ordering The key decides the partition. Events with the same key are consumed in publish order; events without a key (`key: null`) are spread over partitions and have no ordering. Use the aggregate id as the key where order matters. Use the same partitioner in every producer of a topic, otherwise one key can map to different partitions depending on who wrote it. The record key is the event key by default. A `jsonEnvelope` codec with `recordKey: 'none'` writes records without a key, as many existing producers do, and `recordKey: (event) => ...` derives another one. The event key, and the outbox's per-key order, stay as they are: relaybus still publishes events of one key one after the other. But a record without a key is placed on a partition by the producer (round-robin or sticky), so events of one key can land on different partitions and consumers may see them out of order. Keep the event key on the wire wherever consumers rely on per-key order. ## Guarantees - **Publish**: idempotent producer, `acks=-1`; `publish` resolves after the broker ack and rejects with `PublishTimeoutError` after `publishTimeout` (kafkajs would otherwise retry an idempotent send for as long as the broker is unreachable). A rejected publish makes the outbox retry; a timed-out send may still land, so a message can be duplicated, and the idempotent producer plus the consumer's dedup on the event id absorb it. Pass `producerConfig: { retry: { retries } }` (Confluent: `producerConfig: { kafkaJS: { retry: { retries } } }`) to bound client-side retries further. - **Consume**: `eachMessage` awaits the bus (`RelayBus.receive`, one idempotent insert) and the client commits the offset only afterwards. A duplicate message is stored once. - **Handler failure**: retried in-process `processing.maxAttempts` times with backoff and heartbeats (also during the backoff sleeps). If it still fails, the error is rethrown to the client wrapped in a `HandlerFailedError` that keeps the original name and message but has no `cause`; kafkajs classifies a raw `TypeError`, `RangeError`, `ReferenceError` or `SyntaxError` as unrecoverable and would stop the consumer for good. The client does not commit and re-fetches the same record. The record is never skipped, so a persistent outage blocks the partition until the store recovers. That is deliberate: `receive` is a cheap insert, and skipping would lose the event. `isConsuming()` is `false` while any partition is stuck (tracked per partition). - **Dead consumer** (kafkajs; Confluent never gives up on a consumer): the transport does not rely on the client to restart itself. When the client reports a crash with `restart=false`, the transport logs it and reconnects, re-subscribes and re-runs the consumer with bounded backoff (`restart.*`) until it joined again or the transport closed. `isReady()` is `false` meanwhile. We deliberately do not use `pause()` plus `seek()` plus return: the client resolves the offset of a returned `eachMessage` before applying the seek, which can commit past the record if the process dies in that window. - **Malformed or rejected record**: a record that cannot be decoded, or that the bus rejects with any `NonRetryableError` (such as `InvalidEnvelopeError` or `EventValidationError`), is handled by `onInvalidMessage`. The default copies the untouched record to `.dead` with its original headers plus `relaybus-invalid-reason`, `relaybus-source-topic`, `relaybus-source-partition`, `relaybus-source-offset`, `relaybus-error-name` (the error class, such as `InvalidKafkaMessageError`) and, for relaybus errors, `relaybus-error-code` (such as `RELAYBUS_KAFKA_INVALID_MESSAGE`, see [Error codes](https://berkcansavur.github.io/relaybus/concepts/error-codes)), using the same idempotent producer, and commits only after the ack. The log line carries the same `errorName=` and `code=` fields. A failing callback or dead-letter send is retried like a handler failure, so a record is dropped only when you chose `'skip'` or your callback returned. A subscription that consumes its own dead-letter topic is rejected as a loop. - **Start-up**: `subscribe` resolves only after the consumer joined its group. A connect, subscribe or run error, a crash before the first join, or `startupTimeout` rejects it, so `bus.start()` fails instead of reporting a healthy service that consumes nothing. ## Client API used Only these members are called (`KafkaClientLike` and friends in `src/kafka-client.ts`): | Client | Members | | -------------------------- | -------------------------------------------------------------------------------------------------------- | | `kafka` | `producer(config)`, `consumer(config)` | | producer | `connect()`, `send({ topic, acks, messages: [{ key, value, headers }] })`, `disconnect()` | | consumer | `connect()`, `subscribe({ topics, fromBeginning })`, `run({ eachMessage })`, `disconnect()` | | Confluent consumer config | `rebalance_cb(error, assignment)` reports joins (`-175`, assigned) and rebalances (`-174`, revoked) | | `eachMessage` payload | `topic`, `partition`, `message.{ key, value, headers, offset }`, `heartbeat()` | | consumer events (optional) | `consumer.on(name, listener)` and `consumer.events.{ GROUP_JOIN, CRASH, REBALANCING, STOP, DISCONNECT }` | With Confluent, `send` gets no `acks` and `subscribe` no `fromBeginning` (it rejects both; they are set on the producer and consumer instead), and `on`/`events` are not used. `on`/`events` are optional for other kafkajs-like clients: without `GROUP_JOIN` the transport cannot wait for the join or observe crashes, so `subscribe` resolves when `run` returns and `isConsuming()` only reflects processing stalls and `close()`. kafkajs 2.2.4 and `@confluentinc/kafka-javascript` 1.11.0 are verified by the integration tests. ## Reusing the application's producer An application that already has a connected producer, with its SSL and SASL settings, can hand it to the transport instead of letting it open a second connection: ```ts const transport = new KafkaTransport({ producer: ordersService.producer, // already connected, owned by the application client: 'confluent', // when it is a Confluent producer }); ``` Without `kafka` the transport only publishes; give `kafka` and `groupId` as well to subscribe (the consumer stays the transport's own). The transport calls only `send` on the injected producer: it never connects or disconnects it, so it is usable before `bus.start()` and stays open after `bus.stop()`; closing it is the application's job, after the bus stopped. The guarantees hold only if that producer is configured like the transport's own: idempotent, `acks` -1 (all in-sync replicas) and at most 5 in-flight requests (1 keeps retries from reordering a key). For kafkajs that is `kafka.producer({ idempotent: true, maxInFlightRequests: 1 })`, for Confluent `kafka.producer({ kafkaJS: { idempotent: true, acks: -1, maxInFlightRequests: 1 } })`. Because the transport cannot apply `producerConfig` or `maxInFlightRequests` to it, giving them together with `producer` is a configuration error. ## Choosing a client **Use `@confluentinc/kafka-javascript`** (`client: 'confluent'`): it is maintained and built on librdkafka, while `kafkajs` is no longer maintained. Every integration scenario runs against a real broker with both clients (publish and subscribe, retries and redelivery, dead letters, codecs, per-key order across partitions, start-up failure, close, interoperability and two services through the bus); Confluent is tested with 1.11.0, the lowest version the peer range allows. What the transport does differently for Confluent, because its KafkaJS-compatible API differs: - `idempotent`, `maxInFlightRequests` and `acks: -1` go in the producer's `kafkaJS` block, and `groupId` and `fromBeginning` in the consumer's: Confluent rejects them at the top level, in `send` and in `subscribe`. They are also set as librdkafka properties (`enable.idempotence`, `max.in.flight`, `acks`, `group.id`, `auto.offset.reset`), which Confluent applies after the block. Your own `kafkaJS` block and properties are merged below them. - The group join and rebalances are followed through `rebalance_cb` (Confluent has no instrumentation events), so `subscribe` still waits for the join and `isConsuming()` still turns `false` during a rebalance. - Confluent never gives up on a consumer: a failed `eachMessage` makes it seek back and redeliver the record, and it reports no crash. The restart options therefore never apply. - `heartbeat()` is a no-op there; librdkafka heartbeats in the background. Keep the time a record can spend in retries (`maxAttempts * maxBackoff`) well below `max.poll.interval.ms` (5 minutes by default) instead of `sessionTimeout`. - Its producer partitions keys with librdkafka's `murmur2_random`, which librdkafka documents as Java compatible, like kafkajs' `DefaultPartitioner`. `createPartitioner` does not exist there. Moving from kafkajs: create the client with `new KafkaJS.Kafka({ kafkaJS: { brokers, clientId } })`, add `client: 'confluent'`, and move KafkaJS-style settings in `producerConfig` and `consumerConfig` into a `kafkaJS` block (drop `createPartitioner`). The wire format is unchanged. ## Operations - **Topics**: create them up front with the partitions and retention you want. Relying on auto-creation gives broker defaults (and kafkajs subscribes to a missing topic only after auto-creating it). Create the dead letter topics (`.dead` by default) too, or the broker must auto-create them; otherwise the dead-letter send fails and the record keeps blocking its partition (nothing is lost). Monitor the dead-letter topics. - **One subscription per transport**: `subscribe` can be called once; list every topic in that subscription. Create a second `KafkaTransport` (with its own `name` and `groupId`) for a second consumer group. - **Session timeout**: the retry loop heartbeats between attempts. Keep `maxAttempts * maxBackoff` well below the consumer `sessionTimeout` (30 s default). - **Concurrency**: one record at a time per partition (the default of both clients); scale with partitions and service replicas in the same group. With `partitionsConsumedConcurrently > 1` in `consumerConfig` the stall state is tracked per partition. - **Retention vs. inbox**: Kafka may redeliver old records after rebalances or offset resets. The bus deduplicates on the event id for as long as it keeps completed deliveries (`retention.completed`, default 7 days), so keep that longer than your redelivery window. - **Partitioner warning** (kafkajs 2.x only): pass `createPartitioner: Partitioners.DefaultPartitioner` in `producerConfig` to silence the legacy partitioner warning. ## Troubleshooting - `did not join group ... within 30000ms` at start: broker unreachable, wrong credentials, or the group is rebalancing for longer than `startupTimeout`. Check the client's connection settings. - `isConsuming()` stays `false` with `Giving up for now` logs: `RelayBus.receive` keeps failing (database down?). Fix the store; consumption resumes by itself. - `Dead-lettered malformed message` logs: a producer on that topic is not writing CloudEvents binary mode with `content-type: application/json`. Inspect `.dead` and its `relaybus-invalid-reason`. - `Consumer crashed ... restart=false` followed by `Consumer restarted`: the client gave up and the transport recovered it (kafkajs only). Repeated `Restart failed` logs mean the broker or credentials are the problem. - `PublishTimeoutError`: no broker ack within `publishTimeout`; the outbox retries. - Events appear twice in logs but handlers run once: expected, the inbox deduplicates on the id. ## When to use @relaybus/kafka, and when not to Use it to send events between services through Kafka without a dual write between your database and the broker. It is a transport, not a store: it needs `@relaybus/core`, a store and one of the two clients, `@confluentinc/kafka-javascript` (recommended) or `kafkajs`. Documentation: ### relaybus vs alternatives | | What it is | Survives a restart | Written in your database transaction | Choose it when | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | **relaybus** | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries | Yes, events live in your database | Yes | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys | | [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2` | No persistence is documented | No | Simple in-process notifications where losing an event on restart is acceptable | | [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs) | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory | No built-in persistence | No | You want CQRS structure; it can sit on top of an outbox | | [`@nestjs/outbox`](https://github.com/nestjs/outbox) | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores | Yes | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing | | [BullMQ](https://docs.bullmq.io/) | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case | Yes, in Redis | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity | | [pg-boss](https://github.com/timgit/pg-boss) | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction | Yes, in PostgreSQL | Yes, through its database adapters | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports | | Kafka client directly | A distributed log; the idempotent producer and transactions give exactly-once within Kafka | Yes, in Kafka | No, producing to Kafka is a second write | You consume and produce only inside Kafka; add an outbox when a database write must be announced | This table summarises the vendors' own documentation as of October 2026; check their docs for current details. Full comparison and sources: . --- Source: https://berkcansavur.github.io/relaybus/packages/rabbitmq # @relaybus/rabbitmq @relaybus/rabbitmq is the RabbitMQ transport of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: it publishes outbox events with publisher confirms and consumes them with manual acknowledgements and an idempotent inbox. RabbitMQ transport for [relaybus](https://github.com/berkcansavur/relaybus). It publishes events from the outbox with broker confirms and consumes them with manual acknowledgements, so an event is never lost between services: it is either delivered, still waiting in the outbox, or in a dead-letter queue. - **Publish** resolves only after the broker confirmed the message was persisted and routed. - **Consume** acknowledges only after your handler (the relaybus inbox write) succeeded. - **Reconnects** by itself with capped, jittered backoff and restores channels, topology and the consumer. - **Failing handlers** are retried with growing delays (default: up to one hour) through retry queues, so a short outage of your database never dead-letters an event; **poison messages** go to a dead-letter queue immediately. ## Install ```bash pnpm add @relaybus/rabbitmq @relaybus/core amqplib ``` `amqplib` is a peer dependency (`>=0.10`). RabbitMQ 3.13 or 4.x is required (quorum queues). ## Usage ```ts import { RelayBus } from '@relaybus/core'; import { RabbitMqTransport } from '@relaybus/rabbitmq'; const rabbit = new RabbitMqTransport({ name: 'rabbit', url: 'amqp://user:password@broker:5672/vhost?heartbeat=30', exchange: 'events', consumerGroup: 'billing', // queue: relaybus.billing }); const bus = new RelayBus({ store, source: 'billing', transports: [rabbit], // Outbound: committed OrderPaid events are published to the exchange with this routing key. routes: [{ event: OrderPaid, transport: 'rabbit', destination: 'billing.order-paid' }], // Inbound: routing-key patterns bound to this service's queue. subscriptions: [{ transport: 'rabbit', destinations: ['orders.*'] }], }); await bus.start(); // connects; rejects if the broker is unreachable ``` A destination is a routing key when publishing and a binding pattern when subscribing (`orders.*`, `orders.#`, `order.placed.v1`). Readiness probes: `rabbit.isReady()` is true while the transport can publish and, once subscribed, is consuming (connection up, channels open, consumer registered). `rabbit.isConnected()` is the narrower check of the connection alone. relaybus health checks use `isReady()`. ## Options | Option | Default | Description | | -------------------- | ------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | `'rabbitmq'` | Name referenced by routes and subscriptions. | | `url` | - | `amqp://` URL or `{ hostname, port, username, password, vhost, heartbeat, protocol }`. Give exactly one of `url` and `connect`. | | `connect` | - | Factory returning an amqplib connection. Called again after every loss. Use for TLS options or credentials. | | `exchange` | `'relaybus'` | Durable topic exchange, created when missing. | | `prefetch` | `50` (`1` when `ordered`) | Unacknowledged messages held by the consumer; also the handler concurrency. | | `queuePrefix` | `'relaybus'` | Prefix of derived queue names. | | `consumerGroup` | - | Services sharing a group share one queue `.` and split the messages. Required to subscribe unless `queue` is set. | | `queue` | - | Explicit queue name; overrides `queuePrefix` and `consumerGroup`. | | `deadLetterExchange` | `'.dead'` | Direct exchange poison messages are dead-lettered to. | | `deadLetterQueue` | `'.dead'` | Quorum queue that keeps dead-lettered messages. | | `deliveryLimit` | `10` | Quorum queue `x-delivery-limit`: a backstop that makes the broker dead-letter a message that crashes its consumer again and again. Failing handlers do not use it. | | `handlerRetry` | 3 attempts, 200 ms, x2, 5 s cap, equal jitter | Handler attempts, with short backoff, on one delivery before the message is scheduled for delayed redelivery (`RetryPolicyOptions`). | | `redelivery` | 2 s, x2, 5 min cap, equal jitter, `maxAge` `'1h'` | Delayed redelivery after `handlerRetry`: `initialDelay`, `factor`, `maxDelay`, `jitter`, and `maxAge`, how long a message may keep failing before it is dead-lettered. | | `ordered` | `false` | Consume strictly in order: single active consumer, one message at a time, failing messages retried in place. See Ordering. | | `reconnect` | 500 ms, x2, 30 s cap, equal jitter | Reconnect backoff (`initialDelay`, `factor`, `maxDelay`, `jitter`; attempts are unlimited), or `false`. | | `failOnConnectError` | `true` | `connect()` rejects when the broker is unreachable. When `false` it resolves and keeps trying in the background. | | `allowUnroutable` | `false` | When `true`, publishing a message no queue is bound to resolves (the broker discards it) instead of rejecting. | | `publishTimeout` | `'30s'` | How long `publish` waits for the confirm. | | `shutdownTimeout` | `'30s'` | How long `close()` waits for running handlers; when the bus shutdown deadline passes first, the connection's socket is destroyed instead. | | `codec` | `cloudEventsBinary` | The wire format, both ways: `cloudEventsBinary` (CloudEvents binary mode, strict on read), `cloudEventsStructured`, `jsonEnvelope({...})` (your JSON envelope, MassTransit, or the bare payload) or `{ encode, decode }`. See [Interoperating with services that do not use relaybus](#interoperating-with-services-that-do-not-use-relaybus). | | `decode` | none | Deprecated: reads with your function, writes binary mode. Same as `codec: { encode: encodeEnvelope, decode }`; not together with `codec`. | | `logger` | `console` | `log`/`warn`/`error`. | | `random` | `Math.random` | Jitter source, for tests. | ## Wire format CloudEvents 1.0 in binary content mode, following the CloudEvents AMQP binding adapted to RabbitMQ's AMQP 0-9-1: - Body: the JSON payload, `contentType: application/json`, persistent (`deliveryMode: 2`). - Message headers: `cloudEvents:specversion`, `cloudEvents:id`, `cloudEvents:type`, `cloudEvents:source`, `cloudEvents:time` (RFC 3339, millisecond precision), `cloudEvents:datacontenttype`, and when set `cloudEvents:subject` (the event key), `cloudEvents:causationid`, `cloudEvents:correlationid`. - AMQP properties mirror them for broker tooling: `messageId`, `type`, `appId` (source), `correlationId`, `timestamp` (seconds). - Event headers (such as `traceparent`) travel as `relaybus:`, so they cannot collide with CloudEvents attributes or broker headers like `x-death`. The envelope round-trips exactly (`occurredAt` as `Date`, `key`, `causationId`, headers, including a header literally named `__proto__`). On receive the `cloudEvents:` headers win and the AMQP properties are the fallback, so events from other CloudEvents producers work if they follow the binding. The time must be strict RFC 3339 (`2026-01-01T10:20:30.456Z`, offsets allowed); the AMQP `timestamp` is used only when the `cloudEvents:time` header is absent, never to paper over a malformed one. **Payload validation.** The payload must survive `JSON.stringify` unchanged, like on the Kafka transport. `publish` rejects with `UnserializablePayloadError` (a `NonRetryableError`, so the outbox does not retry it forever) for `undefined`, functions, symbols, BigInt, NaN and Infinity, array holes, symbol keys, cycles, and anything JSON would silently convert: `Date`, `Map`, `Set`, typed arrays and class instances. Convert such values to plain data first. Use `null` for "no payload". **AMQP limits.** AMQP 0-9-1 short strings are limited to 255 bytes (UTF-8). The event id (also the AMQP `messageId`, which matches broker returns), every header name (`relaybus:`, so at most 246 bytes for your own names) and the destination (routing key) must fit; otherwise `publish` rejects with `AmqpLimitExceededError` (a `NonRetryableError`) instead of failing on every retry. `type`, `appId` and `correlationId` are mirrored into AMQP properties only when they fit; the `cloudEvents:` headers always carry them, so long values lose only the broker-tooling mirror. ## Interoperating with services that do not use relaybus **relaybus producer, other consumer.** Bind a queue to the exchange and read the message as is: the body is the JSON payload (UTF-8), the `messageId` and `type` properties carry the event id and type, and the `cloudEvents:*` headers carry every attribute. A plain amqplib, Spring AMQP, Go or Python consumer only needs to parse the body; deduplicate on `messageId`, because an outbox delivers at least once. Event headers are prefixed: `traceparent` arrives as `relaybus:traceparent`. **Other formats, both ways.** By default the transport reads only binary-mode CloudEvents; anything else is malformed and dead-lettered, so a misconfigured producer cannot slip events in. The `codec` option sets another wire format for publishing and consuming (one transport is one queue): ```ts import { RabbitMqTransport, cloudEventsStructured, encodeEnvelope, jsonEnvelope, plainJsonDecoder, type RabbitMqCodec, } from '@relaybus/rabbitmq'; // CloudEvents structured mode (`contentType: application/cloudevents+json`); reads binary too. new RabbitMqTransport({ url, consumerGroup: 'billing', codec: cloudEventsStructured }); // MassTransit's JSON envelope. const massTransit = jsonEnvelope({ type: { body: 'messageType.0' }, id: { body: 'messageId' }, payload: { body: 'message' }, correlationId: { body: 'correlationId' }, causationId: { body: 'initiatorId' }, occurredAt: { body: 'sentTime' }, headers: { body: 'headers' }, contentType: 'application/vnd.masstransit+json', }); // The bare JSON payload of a plain amqplib / Spring AMQP producer, the type in a header. const plain = jsonEnvelope({ type: { header: 'event-type' }, id: 'message-id-property', // the AMQP messageId, or { header: 'message-id' } payload: 'whole-body', }); // Your own: binary mode out, plain JSON and CloudEvents in. const custom: RabbitMqCodec = { encode: encodeEnvelope, decode: plainJsonDecoder({ type: (message) => message.properties.type as string | undefined }), }; ``` | Codec | Writes | Reads | | ----------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | | `cloudEventsBinary` | Binary mode, as above. Default, byte for byte unchanged. | Binary mode only (AMQP properties as fallback). | | `cloudEventsStructured` | One `application/cloudevents+json` document, binary-mode AMQP properties. | Structured and binary mode (`decodeCloudEvent`). | | `jsonEnvelope(mapping)` | The mapped JSON body and headers, binary-mode AMQP properties. | The same mapping. A missing type, mapped id or payload is malformed. | | `{ encode, decode }` | Whatever `encode` returns (`{ content, options }`; set `messageId`). | Whatever `decode` returns; missing fields are filled as for a decoder (table below). | `jsonEnvelope` mappings name a body path (`{ body: 'meta.correlationId' }`, `'messageType.0'` indexes an array) or a header (`{ header: 'event-type' }`) for `type`, `id`, `payload` (or `'whole-body'`), `key`, `source`, `occurredAt`, `correlationId`, `causationId` and `headers`, plus a `contentType`. `type`, `id` and `payload` are required by the types; `id: 'message-id-property'` uses the `messageId` property, which the codec always writes. Unmapped, `source` is the `appId` property and `occurredAt` the `timestamp` property (whole seconds). Event headers that are message headers keep their names: a name the mapping uses, or one starting with `x-death`, `x-first-death-`, `x-last-death-` or `x-relaybus-`, is refused with `RabbitMqUnencodableEventError` (a `NonRetryableError`). The [interoperability recipe](https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services) has a full table. Decoded payloads go through the event's schema (`defineEvent(type, { schema })`) as relaybus's own do; a payload that fails it makes the delivery dead with `EventValidationError`. The decoders below remain available for a custom codec's `decode`, and for the deprecated `decode` option: | Decoder | Accepts | | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | default | Binary-mode CloudEvents only, as described above. Unchanged. | | `decodeCloudEvent` | Binary mode as the default, plus structured mode: a JSON body with `specversion` `1.0`, `id`, `type`, `source`, optional `time` (else the `timestamp` property, else receipt), `subject` (the key), `data` (the payload; `data_base64` is refused). | | `plainJsonDecoder(opts)` | Any JSON body. A message that is a CloudEvent anyway (a `cloudEvents:specversion` header or a structured content type) goes through `decodeCloudEvent`, so relaybus and foreign producers can feed one queue. | | your function | `(message) => ({ id, type, payload, ...optional })`. Missing fields are filled: `source` = `appId` (else routing key), `key` = null, `occurredAt` = `timestamp` property (else receipt), `headers` = `{}`. | `plainJsonDecoder` options (each extractor gets `{ routingKey, properties, headers, payload }`; an extractor returning `undefined` or `''` selects the default): | Option | Default | Meaning | | ------------ | -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | required | A fixed event type, or an extractor such as `fromHeader('event-type')`, `fromPayloadField('type')` or `(m) => m.properties.type`. | | `id` | the `messageId` property | Inbox dedup id. A message with neither an extracted id nor a `messageId` is malformed: AMQP has no other identity stable across redeliveries. | | `key` | none | Ordering key, for example `fromHeader('customer')`. | | `source` | the `appId` property, else routing key | Fixed text or an extractor. | | `occurredAt` | the `timestamp` property, else receipt | Extractor returning a `Date`. | String message headers become event headers, except the broker's and the transport's own (`x-first-death-*`, `x-last-death-*`, `x-relaybus-*`; non-string values such as `x-death` are skipped). **Guarantees stay the same.** Decoding runs again on every redelivery (also after a retry-queue round trip, which keeps `messageId` and restores the original routing key from the `x-relaybus-routing-key` header it adds to retry copies) and must be deterministic; the inbox deduplicates on the id, so a redelivered message is stored and handled once. Under every codec, a message that cannot be decoded (not JSON, no id, no type, an invalid type, an extractor returning a non-string) and anything a custom `decode` throws are dead-lettered (`x-relaybus-dead-letter-reason: malformed message`, the detail is logged) and the consumer keeps running. Messages are handled in queue order only with `ordered: true`. **Give producers a `messageId`.** Spring AMQP generates one when the message converter has `setCreateMessageIds(true)`; amqplib takes the `messageId` publish option. Without it, configure `id` to read a business id from a header or the payload. ## Guarantees - **No silent loss on publish.** `publish` rejects (never resolves) when the transport is disconnected or closed (`RabbitMqUnavailableError`), the broker nacks (`RabbitMqPublishRejectedError`), the confirm does not arrive within `publishTimeout` (`RabbitMqUnavailableError`) or no queue is bound to the routing key (`UnroutableMessageError`). The outbox keeps the delivery and retries it. A rejection after the broker already stored the message (lost confirm, timeout) leads to a duplicate on retry; consumers deduplicate on the event id. - **Mandatory publishing trade-off.** Messages are published with the AMQP `mandatory` flag so an event nobody is bound to is reported instead of vanishing. The cost is that a publisher fails until at least one consumer queue is bound to the routing key (start consumers first, or declare queues in advance). Set `allowUnroutable: true` for fire-and-forget topics where "no subscriber" is fine. - **Acknowledge after handling.** A message is acknowledged after `onMessage` resolved, which in relaybus means the event is in the inbox. A crash before that redelivers it; the inbox drops the duplicate. - **Outages are waited out, not burned through.** A failing handler is retried on the same delivery with short backoff (`handlerRetry`). If it still fails, a copy of the message is published to a retry queue and the original is acknowledged **only after the broker confirmed the copy** (publisher confirms, mandatory flag), so the message is always in exactly one durable place at least. When the copy's delay is over, the broker moves it back to the main queue. Delays grow by `redelivery.factor` up to `redelivery.maxDelay`, with jitter. With the defaults a message is retried at about 2, 4, 8, 16, 32, 64, 128, 256 s, then every 5 min, and dead-lettered after 1 h of failing (`redelivery.maxAge`), about 18 redeliveries of 3 handler attempts each. A multi-minute database outage therefore delays events and loses nothing. If the copy cannot be confirmed, the original is requeued (after the same delay), never acknowledged. - **Delay tiers.** Each distinct delay is its own quorum retry queue (`.retry.`, nine with the defaults) with a queue-level TTL, so messages in one queue expire in arrival order; one queue with per-message TTLs would let a long delay block shorter ones. The jittered delay is rounded to the tier below or above with the probability that keeps its average, so a burst of failures still spreads across tiers. A retry copy carries `x-relaybus-redeliveries` and `x-relaybus-first-failure-at`, which makes the window independent of which replica handles it. - **Immediate dead-lettering** for messages that can never succeed: bodies that are not valid CloudEvents (logged at `error`) and handler errors that are `NonRetryableError`s, such as `InvalidEnvelopeError` from `bus.receive`. The consumer publishes a confirmed copy to the dead-letter exchange, then acknowledges the original; the copy keeps the message headers, drops the retry state (so replaying it starts a fresh retry window) and adds `x-relaybus-dead-letter-reason`, `x-relaybus-dead-letter-error-name` (the error class, such as `MalformedMessageError`) and, for relaybus errors, `x-relaybus-dead-letter-error-code` (such as `RELAYBUS_RABBITMQ_MALFORMED_MESSAGE`, see [Error codes](https://berkcansavur.github.io/relaybus/concepts/error-codes)). The `error` log lines carry the same `errorName=` and `code=` fields. If that publish fails the consumer rejects the message without requeue and the broker dead-letters it (at-least-once strategy), so it still ends in `.dead`. - **Broker-level backstop.** `x-delivery-limit` (`deliveryLimit`) dead-letters a message that keeps coming back because the consumer or its channel dies while holding it. - **Reconnection.** On connection loss, a closed channel or a cancelled consumer the transport drops the connection, waits with capped, jittered backoff, reconnects and re-declares the exchange, queues, bindings and consumer. Handlers still running during the loss finish; their acks fail (logged) and the broker redelivers. - **Ordering.** By default order is **not** preserved: `prefetch` handlers run concurrently, several replicas compete on the queue, and a delayed redelivery overtakes later messages, so `EventEnvelope.key` order does not hold across the broker. Handlers must tolerate reordering (versioned state, "ignore older" checks). When per-key order matters, set `ordered: true`: the queue gets `x-single-active-consumer` (one replica consumes, the others wait as standby and take over when it goes away), prefetch is 1, and a failing message is retried **in place** with the `redelivery` delays instead of moving behind newer ones, which blocks the queue until it succeeds or `redelivery.maxAge` passes and it is dead-lettered. The cost is throughput: one handler call at a time for the whole consumer group, and one stuck message stalls everything behind it. Order is queue-wide (a superset of per-key order) and holds for messages published by one producer on one connection; the broker cannot order across publishers. ## Topology For a subscription with `queue = relaybus.billing`, `exchange = relaybus`: | Object | Declaration | | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `relaybus` exchange | topic, durable | | `relaybus.dead` exchange | direct, durable | | `relaybus.billing.dead` queue | quorum, durable, bound to `relaybus.dead` with key `relaybus.billing` | | `relaybus.billing.retry.` | one quorum queue per delay tier (not when `ordered`): `x-message-ttl`, dead-lettered through the default exchange to `relaybus.billing`, `x-dead-letter-strategy: at-least-once`, `x-overflow: reject-publish` | | `relaybus.billing` queue | quorum, durable, `x-delivery-limit`, `x-dead-letter-exchange`, `x-dead-letter-routing-key`, `x-dead-letter-strategy: at-least-once`, `x-overflow: reject-publish`, `x-single-active-consumer` when `ordered`; bound to the exchange once per destination | `at-least-once` dead-lettering makes the quorum queue keep a message until the dead-letter target confirmed it, so expiry, rejection and the delivery limit cannot lose messages while the target is unavailable. RabbitMQ requires `reject-publish` overflow with it. Queues are unbounded by default, so this only matters if you add a `max-length` policy: then publishers are refused instead of the oldest message being dropped. Declarations are idempotent. Redeclaring a queue with different arguments makes the broker answer `PRECONDITION_FAILED` and `subscribe`/startup rejects; the transport never alters an existing queue. Removing a destination from a subscription does not unbind it. ### Migrating an existing queue Queues created by a pre-release version of this package lack `x-dead-letter-strategy` and `x-overflow`, and queues created with another `ordered` setting differ in `x-single-active-consumer`. RabbitMQ cannot change these arguments in place, so either: 1. use a new `queue` / `consumerGroup` name, let both queues receive events (the new one is bound by the new deployment), and delete the old queue once it is empty; or 2. stop the consumers, drain or shovel the old queue (`rabbitmqctl shovel` / the Shovel plugin) into the exchange, delete it (`rabbitmqctl delete_queue `), and start the new version. The dead-letter queue and exchange are unchanged and need no migration. ## Operational notes - **Heartbeats.** When the URL carries no `heartbeat`, amqplib adopts the interval the broker proposes (RabbitMQ defaults to 60 s), in both the 0.10 and the 2.x range, so a dead TCP connection is noticed after roughly two intervals. Set `?heartbeat=30` (or `heartbeat` in the settings object) to pick a shorter interval; the lower of the two values is negotiated. Beware `heartbeat=0`: from amqplib 2.0 it explicitly **disables** heartbeats (before 2.0 it meant "use the server's value"), and without heartbeats only `publishTimeout` catches a silently dead connection for publishers while consumers can hang. Do not set it to 0. - **Dead-letter queue.** Monitor `.dead`. To replay, move messages back to the exchange with the Shovel plugin or a script; the headers (`cloudEvents:*`) are intact and the inbox deduplicates by id. - **One subscription per transport.** A transport owns one consumer group's queue. To consume several groups from one process create several `RabbitMqTransport` instances with different `name` and `consumerGroup`. - **Startup.** `failOnConnectError: true` (default) makes `bus.start()` fail fast when the broker is down, so orchestrators restart the pod. Use `false` to boot degraded; publishes are rejected and retried by the outbox until the broker is back. - **TLS.** Pass `connect: () => amqplib.connect('amqps://...', { ca: [...] })`. - **Consumer scaling.** Add replicas with the same `consumerGroup`; they compete on one queue. ## Troubleshooting | Symptom | Cause and fix | | ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `UnroutableMessageError` | No queue is bound to the routing key. Start the consuming service first, check the destination, or set `allowUnroutable`. | | `PRECONDITION_FAILED` on start | A queue or exchange exists with different arguments or type. Align the options or delete the old object. | | `Not connected to the broker` | Broker unreachable or reconnecting; the outbox retries. Check `isConnected()` and the `[ConnectionSupervisor]` log lines. | | Messages pile up in `.dead` | Handler errors, malformed messages or a dependency that stayed down past `redelivery.maxAge`; the `error` log lines and the `x-relaybus-dead-letter-reason` header name the cause. | | `UnserializablePayloadError` / `AmqpLimitExceededError` | The event can never be published as is: fix the payload (plain JSON data only) or shorten the id, header name or routing key. | | Events arrive out of order | Expected without `ordered: true`; see Ordering. | | `isReady()` is false but `isConnected()` is true | The consumer was cancelled (queue deleted, node failover) and is being restored, or the publisher channel closed. | | Publishes hang until `publishTimeout` | Broker is blocked (memory or disk alarm) or the connection died without heartbeats. | | `Needs 'consumerGroup' or 'queue' to subscribe` | Set `consumerGroup` (or `queue`) on the transport that subscribes. | ## When to use @relaybus/rabbitmq, and when not to Use it to send events between services through RabbitMQ with publisher confirms, retry queues and dead-letter queues. It needs RabbitMQ 3.13 or 4.x (quorum queues) and does not preserve order unless you set `ordered: true`. Documentation: ### relaybus vs alternatives | | What it is | Survives a restart | Written in your database transaction | Choose it when | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | **relaybus** | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries | Yes, events live in your database | Yes | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys | | [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2` | No persistence is documented | No | Simple in-process notifications where losing an event on restart is acceptable | | [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs) | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory | No built-in persistence | No | You want CQRS structure; it can sit on top of an outbox | | [`@nestjs/outbox`](https://github.com/nestjs/outbox) | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores | Yes | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing | | [BullMQ](https://docs.bullmq.io/) | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case | Yes, in Redis | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity | | [pg-boss](https://github.com/timgit/pg-boss) | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction | Yes, in PostgreSQL | Yes, through its database adapters | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports | | Kafka client directly | A distributed log; the idempotent producer and transactions give exactly-once within Kafka | Yes, in Kafka | No, producing to Kafka is a second write | You consume and produce only inside Kafka; add an outbox when a database write must be announced | This table summarises the vendors' own documentation as of October 2026; check their docs for current details. Full comparison and sources: . --- Source: https://berkcansavur.github.io/relaybus/packages/testing # @relaybus/testing @relaybus/testing is the test toolkit of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: it provides the EventStore contract suite, a `FakeClock` and an in-memory transport for deterministic tests of retries and delivery. Test utilities for relaybus. ```sh pnpm add -D @relaybus/testing vitest # or jest: see the codec contract below ``` ## Store contract suite Every `EventStore` must pass it unchanged against a real database: ```ts import { eventStoreContract } from '@relaybus/testing'; eventStoreContract('MyEventStore', { createStore: async () => { const store = new MyEventStore(await connect()); await store.setup(); return store; }, destroyStore: async (store) => store.close(), // a second instance on the same database with its own connections createPeerStore: async () => new MyEventStore(await connect()), }); ``` It covers transactional visibility, idempotent appends (a handler that already completed is never delivered again), leasing and fencing, batched completion (`completeMany`), lost leases (redelivery, attempt counting for crashed workers, poison deliveries), retries and dead letters, release, relinquish, per-key ordering in commit order, starvation, purge, statistics, data fidelity and cross-instance claim races. A store may lease groups of deliveries (lanes) instead of single deliveries, so the contract never assumes that two claimers see each other's unclaimed work: a claimer that keeps claiming drains the workload. Pass `keepsCompletedRecords: true` when the store keeps a completion history; run the suite once per mode the store supports. ## Codec contract Before adopting a wire format that other services already use, prove that a codec writes and reads it exactly. Give `kafkaCodecContract` or `rabbitMqCodecContract` captured messages and the events they carry; each case is one test that checks `encode(event)` against the captured message (value or body byte for byte or as canonical JSON, key, headers, timestamp or properties) and that decoding the captured message gives the event: ```ts import { kafkaCodecContract } from '@relaybus/testing'; import { decodeKafkaRecord } from '@relaybus/kafka'; kafkaCodecContract('orders format', { encode: orderCodec.encode, decode: (record) => decodeKafkaRecord(orderCodec, record), compare: 'bytes', // or 'json' to ignore key order and spacing ignoreHeaders: ['traceparent'], cases: [ { name: 'order placed', envelope, // the event record: { topic: 'orders', key: 'order-1', headers: {}, value: '{"orderId":"1"}' }, // the captured record // only: 'decode', // for a message only other producers write }, ], }); ``` `rabbitMqCodecContract` takes the same shape with `decodeRabbitMqMessage` from `@relaybus/rabbitmq` and captured messages (routing key, properties, headers, body). `kafkaCodecMismatches` and `rabbitMqCodecMismatches` return the differences as text lines instead of registering tests. Compared byte for byte, the decoded payload must also keep the expected key order. The contract does not need Vitest: ```ts // Jest: registers with Jest's describe and it globals import { kafkaCodecContract } from '@relaybus/testing/jest'; // Any runner, or a plain script: no test framework imported import { assertKafkaCodecCase, // throws CodecContractError listing the mismatches registerKafkaCodecContract, // registerKafkaCodecContract({ describe, it }, { name, options }) } from '@relaybus/testing/codec-contract'; ``` The RabbitMQ counterparts are `assertRabbitMqCodecCase` and `registerRabbitMqCodecContract`. ## FakeClock ```ts const clock = new FakeClock('2026-01-01T00:00:00Z'); const bus = new RelayBus({ store, clock }); clock.advance(60_000); // make retries due await bus.drain(); ``` Timers (polling, heartbeats, handler timeouts, retention) stay real by default. With `new FakeClock(start, { controlTimers: true })` the clock owns the timers of the bus that uses it: they run inside `clock.advance(ms)` when their time comes, and `clock.pendingTimers` counts the ones still waiting. Any `Clock` can do the same by implementing the optional `schedule` method. ## InMemoryTransport Records publishes, simulates outages with `failNextPublishes(n)` and delivers JSON copies to subscribers with `deliver(destination, envelope, maxRedeliveries)`. ## When to use @relaybus/testing, and when not to Use it in your test suite for fast, deterministic tests, or to prove that a custom `EventStore` satisfies the contract. It is a development dependency. The store contract, the main entry and `kafkaCodecContract` / `rabbitMqCodecContract` need `vitest` 3 or newer (an optional peer); `@relaybus/testing/jest` and `@relaybus/testing/codec-contract` do not. Documentation: ### relaybus vs alternatives | | What it is | Survives a restart | Written in your database transaction | Choose it when | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | **relaybus** | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries | Yes, events live in your database | Yes | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys | | [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2` | No persistence is documented | No | Simple in-process notifications where losing an event on restart is acceptable | | [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs) | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory | No built-in persistence | No | You want CQRS structure; it can sit on top of an outbox | | [`@nestjs/outbox`](https://github.com/nestjs/outbox) | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores | Yes | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing | | [BullMQ](https://docs.bullmq.io/) | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case | Yes, in Redis | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity | | [pg-boss](https://github.com/timgit/pg-boss) | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction | Yes, in PostgreSQL | Yes, through its database adapters | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports | | Kafka client directly | A distributed log; the idempotent producer and transactions give exactly-once within Kafka | Yes, in Kafka | No, producing to Kafka is a second write | You consume and produce only inside Kafka; add an outbox when a database write must be announced | This table summarises the vendors' own documentation as of October 2026; check their docs for current details. Full comparison and sources: . ## License MIT --- Source: https://berkcansavur.github.io/relaybus/packages/eslint-plugin # @relaybus/eslint-plugin @relaybus/eslint-plugin is the ESLint plugin of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: it keeps outbox transactions short and events atomic with your writes by flagging calls to other services inside a transaction and `publish` calls outside one. ```sh pnpm add -D @relaybus/eslint-plugin ``` ESLint 9 or newer, flat config. For TypeScript files with decorators use the `typescript-eslint` parser (type information is not needed). ```js // eslint.config.js import tseslint from 'typescript-eslint'; import relaybus from '@relaybus/eslint-plugin'; export default tseslint.config(...tseslint.configs.recommended, relaybus.configs.recommended); ``` `recommended` turns both rules on at `error` under the plugin name `@relaybus`. ## Where a transaction is Both rules agree on what runs inside a transaction: - a method decorated with `@RelayBusTransaction()`; - a handler that runs in a transaction: `@RelayBusConsumer(Event, { transactional: true })`, an `on` method of a `@RelayBusCatalogConsumer(Catalog, { transactional: true })` class (also under the deprecated names `@Subscribe` and `@EventConsumer`), and `bus.subscribe(Event, handler, { transactional: true })`; - the callback of `x.transaction(...)`, `x.$transaction(...)` or `x.withTransaction(...)` (`bus.transaction`, TypeORM, Knex, Sequelize, Prisma, MongoDB sessions); - any method of the same class called through `this.method()` from one of the above, transitively. ## `@relaybus/no-external-call-in-transaction` Reports calls to other services inside a transaction: `fetch`, `axios` (`axios(...)`, `axios.get(...)`, any method), `got`, `http.request` / `http.get` / `https.*` of Node.js, `undici` (`request`, `fetch`, `stream`, `pipeline`), and `HttpService` of `@nestjs/axios` (`this.httpService.get(...)`, `this.http.post(...)`, `httpService.axiosRef.get(...)`). Imports, renamed imports and `require` are followed. A transaction holds its row locks and a database connection until it ends. An HTTP call inside it makes every other writer of those rows wait for the remote service, and relaybus rolls the transaction back with `TransactionTimeoutError` once it outlives `transactionTimeout`. ```ts // Reported @RelayBusTransaction() async place(order: Order): Promise { const price = await this.httpService.axiosRef.get(`/prices/${order.sku}`); await this.orders.insert({ ...order, price: price.data }); await this.bus.publish(new OrderPlaced({ orderId: order.id })); } // Fixed: call before the transaction, and react to the event after the commit async place(order: Order): Promise { const price = await this.httpService.axiosRef.get(`/prices/${order.sku}`); await this.save({ ...order, price: price.data }); } @RelayBusTransaction() private async save(order: PricedOrder): Promise { await this.orders.insert(order); await this.bus.publish(new OrderPlaced({ orderId: order.id })); } ``` ## `@relaybus/publish-requires-transaction` Reports `publish` / `publishAll` on a relaybus bus or catalog publisher outside a transaction without `{ transaction }`. Such a publish is not atomic with your writes and throws `TransactionRequiredError` at runtime (`transactionMode: 'required'`, the default). Options that the rule cannot see into (a variable, a spread) are trusted. The publishers are recognised by name: the object's last name ends in `bus`, `Bus`, `publisher` or `Publisher` (`this.bus`, `eventBus`, `orderPublisher`). Set your own pattern when you name them differently: ```js { rules: { '@relaybus/publish-requires-transaction': ['error', { publishers: '^(bus|orders)$' }] } } ``` ## Limits The rules read one file and know names, not types: a call to a service hidden behind your own wrapper function, or a transaction opened in another file, is not seen. The runtime checks stay in place behind them (`TransactionRequiredError`, `transactionTimeout`). Documentation: https://berkcansavur.github.io/relaybus/recipes/transactions-in-nestjs ## License MIT --- Source: https://berkcansavur.github.io/relaybus/packages/otel # @relaybus/otel @relaybus/otel is the OpenTelemetry integration of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: messaging metrics after the OpenTelemetry semantic conventions, outbox backlog gauges, and one trace from `publish` in one service to the handler in another. ```sh pnpm add @relaybus/otel @opentelemetry/api ``` It depends only on `@opentelemetry/api` (peer, `^1.3.0`); the SDK, exporters and collector are yours. ```ts import { relayBusOpenTelemetry } from '@relaybus/otel'; const otel = relayBusOpenTelemetry(); // global meter and tracer providers, W3C propagation const bus = new RelayBus({ store, observers: [otel.observer], // metrics tracer: otel.tracer, // spans + traceparent/tracestate in event headers backlogInterval: '30s', // feeds the backlog gauges }); ``` In NestJS pass the same three options to `RelayBusModule.forRoot`. | Option | Default | What it does | | ---------------- | ---------------------------- | --------------------------------------------------------------------------------------------------------- | | `meterProvider` | `metrics.getMeterProvider()` | Where the metrics are created. | | `tracerProvider` | `trace.getTracerProvider()` | Where the spans are created. | | `propagator` | W3C Trace Context (built in) | Writes and reads the trace context of events. Pass `propagation` from `@opentelemetry/api` for the SDK's. | ## Traces | Span | Kind | Parent | In which process | | ---------------------- | -------- | ---------------------------------------------------- | ------------------------------------ | | `publish ` | producer | the caller's active span | the publisher, inside `publish` | | `send ` | producer | the publish span (from the stored event), linked | the publisher, when a route sends it | | `process ` | consumer | the context the event arrived with (send or publish) | the consumer, around the handler | The publish span's context is stored with the event (`traceparent`, `tracestate` headers), so the trace survives the outbox, retries and restarts. A route sends the event with the send span's context; the consumer's `process` span continues from it, and whatever the handler does or publishes joins that span. Without a tracer the headers are still readable with `traceContextOf` from `@relaybus/core`. ## Metrics Durations are in seconds. `error.type` is the relaybus error code (`RELAYBUS_*`) when there is one, else the error's class name; it is present only on failures. | Metric | Type | Unit | Attributes | | ------------------------------------- | --------- | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `messaging.client.operation.duration` | histogram | `s` | `messaging.system`, `messaging.operation.name` (`publish`, `claim`, `send`, `receive`), `messaging.operation.type`, `messaging.destination.name`, `error.type` | | `messaging.client.sent.messages` | counter | `{message}` | `messaging.system`, `messaging.destination.name`, `messaging.operation.name`, `messaging.operation.type`, `error.type` | | `messaging.client.consumed.messages` | counter | `{message}` | as above, plus `relaybus.receive.outcome` (`accepted`, `duplicate`, `ignored`, `invalid`, `error`) | | `messaging.process.duration` | histogram | `s` | `messaging.system`, `messaging.destination.name` (event type), `messaging.consumer.group.name` (handler id), `messaging.operation.*`, `error.type` | | `relaybus.messages.invalid` | counter | `{message}` | `messaging.system`, `messaging.destination.name`, `error.type` (inbound messages a transport could not decode; they never reach `receive`) | | `relaybus.events.published` | counter | `{event}` | `messaging.system`, `messaging.destination.name` (event type), `error.type` | | `relaybus.claim.deliveries` | histogram | `{delivery}` | `messaging.system`, `messaging.operation.*` | | `relaybus.claim.lanes` | histogram | `{lane}` | `messaging.system`, `messaging.operation.*` | | `relaybus.end_to_end.latency` | histogram | `s` | `messaging.system`, `messaging.destination.name`, `messaging.consumer.group.name`, `relaybus.delivery.kind` (`handler`, `route`) | | `relaybus.deliveries.retried` | counter | `{delivery}` | `messaging.system`, `messaging.destination.name`, `messaging.consumer.group.name`, `error.type` | | `relaybus.deliveries.dead_lettered` | counter | `{delivery}` | as above | | `relaybus.lease.lost` | counter | `{delivery}` | `messaging.system`, `messaging.destination.name`, `messaging.consumer.group.name` | | `relaybus.deliveries.pending` | gauge | `{delivery}` | none (needs `backlogInterval`) | | `relaybus.deliveries.oldest_age` | gauge | `s` | none (needs `backlogInterval`) | | `relaybus.failures` | counter | `{failure}` | `relaybus.failure.stage` (`claim`, `complete`, `purge`, `backlog`, ...), `error.type` | `messaging.system` is `relaybus` for work on the outbox (publish, claim, process) and the transport's system (`kafka`, `rabbitmq`) for sends and receives. The full guide, with a collector, Prometheus and alert rules, is the [Observability](https://berkcansavur.github.io/relaybus/concepts/observability) page. ## License MIT --- Source: https://berkcansavur.github.io/relaybus/recipes/transactions-in-nestjs # Transactions in NestJS The outbox only works when the event is written in the same transaction as your business data. This page shows the NestJS way to get that without threading a transaction handle through every call, how repositories join the same transaction, how relaybus keeps transactions short, and the lint rules that catch mistakes before review does. ## Before: the transaction is threaded by hand ```ts @Injectable() export class OrderService { constructor( private readonly bus: RelayBus, private readonly orders: OrderRepository, ) {} async place(orderId: string): Promise { await this.bus.transaction(async (tx) => { await this.orders.insert(tx, orderId); // every repository method takes tx await this.bus.publish(new OrderPlaced({ orderId }), { transaction: tx }); }); } } ``` ## After: `@RelayBusTransaction()` ```ts @Injectable() export class OrderService { constructor( private readonly bus: RelayBus, private readonly orders: OrderRepository, ) {} @RelayBusTransaction() async place(orderId: string): Promise { await this.orders.insert(orderId); await this.bus.publish(new OrderPlaced({ orderId })); // joins the transaction } } @Injectable() export class OrderRepository { constructor(private readonly transactions: RelayBusTransactionHost) {} async insert(orderId: string): Promise { await this.transactions.current().query('INSERT INTO orders (id) VALUES ($1)', [orderId]); } } ``` - The method commits when it resolves and rolls back when it throws. - The decorator replaces the method itself, so `this.place()` called from another method of the class is transactional too; interceptor-based decorators miss such self-calls. - A decorated method called inside another one (or inside `bus.transaction`, a transactional handler, or a transaction `transactionResolver` finds) joins that transaction. - The class does not inject anything for the decorator: the bus is the one of the application's `RelayBusModule`. Without a running module the call throws a `ConfigurationError` saying so. - Only async methods compile: on a method that does not return a Promise, TypeScript reports `'@RelayBusTransaction() needs a method that returns a Promise: make the method async'`. - `@RelayBusTransaction({ isolationLevel: 'serializable' })` runs the method at that level and reruns it after a serialization failure; see [Serializable transactions](https://berkcansavur.github.io/relaybus/recipes/serializable-transactions). A method that asks for a level can join only a transaction relaybus opened at the same level. ## Repositories with your data library `RelayBusTransactionHost` is typed with the store's transaction handle. `current()` throws `TransactionRequiredError` outside a transaction; `find()` returns `undefined`. Outside NestJS, `bus.currentTransaction()` does the same as `find()`. | Library | Store / executor | `T` | A repository writes with | | ------------------ | ----------------------------------- | -------------------------- | --------------------------------------------- | | `pg` | `new PostgresEventStore({ pool })` | `PoolClient` | `tx.query(sql, values)` | | Kysely | `createKyselyExecutor(db)` | `Transaction` | `tx.insertInto('orders').values(o).execute()` | | TypeORM | `createTypeOrmExecutor(dataSource)` | `EntityManager` | `tx.getRepository(Order).save(order)` | | Prisma | `createPrismaExecutor(prisma)` | `Prisma.TransactionClient` | `tx.order.create({ data })` | | MongoDB / Mongoose | `new MongoEventStore({ client })` | `ClientSession` | `Order.create([doc], { session: tx })` | ### Next to `@nestjs-cls/transactional` If your services already use `@Transactional()` from `@nestjs-cls/transactional`, keep it and let relaybus find its transaction through `transactionResolver`. A `@RelayBusTransaction()` method called inside a `@Transactional()` one then joins that transaction instead of opening a second one, and `publish` writes into it: ```ts RelayBusModule.forRootAsync({ inject: [TransactionHost], useFactory: (txHost: TransactionHost) => ({ store, transactionResolver: () => (txHost.isTransactionActive() ? txHost.tx : undefined), }), }); ``` ## Time budget Every relaybus transaction is held to `transactionTimeout` (default `'5s'`), enforced by the database where it can be: - **PostgreSQL:** `statement_timeout`, and with the `pg` executor also `idle_in_transaction_session_timeout` and, on PostgreSQL 17+, `transaction_timeout`, all set for the transaction only. A transaction left idle by a slow HTTP call is ended by the server and its locks are released. - **MongoDB:** `maxCommitTimeMS` on the commit. A transaction that still runs past the budget is rolled back before its commit. The caller gets a `TransactionTimeoutError` (`RELAYBUS_TRANSACTION_TIMEOUT`), with the database's error as `cause`, whose message names the usual reason: a call to another service inside the transaction. Move the call out instead of raising the budget. ## Lint rules `@relaybus/eslint-plugin` checks both halves at `error` level: ```sh pnpm add -D @relaybus/eslint-plugin ``` ```js // eslint.config.js import tseslint from 'typescript-eslint'; import relaybus from '@relaybus/eslint-plugin'; export default tseslint.config(...tseslint.configs.recommended, relaybus.configs.recommended); ``` - `@relaybus/no-external-call-in-transaction` reports `fetch`, `axios`, `got`, Node's `http` / `https`, `undici` and `@nestjs/axios`'s `HttpService` inside `@RelayBusTransaction()` methods, transactional handlers, `.transaction(...)` callbacks, and the methods they call through `this`. - `@relaybus/publish-requires-transaction` reports `bus.publish` / `publishAll` outside all of those without `{ transaction }`. ```ts @RelayBusTransaction() async place(order: Order): Promise { await this.httpService.axiosRef.post('/fraud-check', order); // no-external-call-in-transaction await this.orders.insert(order); await this.bus.publish(new OrderPlaced({ orderId: order.id })); } async cancel(orderId: string): Promise { await this.bus.publish(new OrderCancelled({ orderId })); // publish-requires-transaction } ``` See the [package page](https://berkcansavur.github.io/relaybus/packages/eslint-plugin) for what each rule recognises and its options. --- Source: https://berkcansavur.github.io/relaybus/recipes/serializable-transactions # Serializable transactions `bus.transaction` and `@RelayBusTransaction()` run at the database default (READ COMMITTED on PostgreSQL) unless you ask for another level. This page shows when to ask for `serializable`, what relaybus does when PostgreSQL aborts a transaction because of a concurrent one, and what your code must not do inside such a transaction. ## The problem: write skew Two requests each count a user's active reservations, check a limit of one, and insert: ```ts @RelayBusTransaction() async reserve(userId: string): Promise { const active = await this.reservations.countActive(userId); // SELECT count(*) ... if (active >= 1) throw new LimitReachedError(userId); await this.reservations.insert(userId); await this.bus.publish(new ReservationCreated({ userId })); } ``` At READ COMMITTED both requests count zero, both pass the check and both insert: the limit is exceeded, although every statement was correct. The usual fixes take a pessimistic lock (`pg_advisory_xact_lock`, `SELECT ... FOR UPDATE`) that makes every request for that user wait for the one before it. ## The fix: `isolationLevel: 'serializable'` ```ts @RelayBusTransaction({ isolationLevel: 'serializable' }) async reserve(userId: string): Promise { /* the same body */ } ``` Under SERIALIZABLE, PostgreSQL lets both requests run without blocking each other and aborts one of them with SQLSTATE `40001` (`could not serialize access`) when the outcome could not have been produced by running them one after the other. relaybus then **runs the aborted transaction again**, in a new one. This time the count sees the other request's committed row, and your own check rejects it with `LimitReachedError`, which is not a serialization failure and is thrown to the caller as it is. The same with `bus.transaction`: ```ts await bus.transaction( async (tx) => { /* ... */ }, { isolationLevel: 'serializable', maxAttempts: 5 }, ); ``` ## What runs again, and what that means for your code At `'repeatable read'` and `'serializable'`, a serialization failure (SQLSTATE `40001`) during the transaction or at its commit rolls the attempt back and runs `work` again in a new transaction, up to `maxAttempts` attempts in all (default 3). When the last attempt fails too, its error is thrown unchanged. Other errors, the `TransactionTimeoutError` of a budget overrun and every error at `'read committed'` are never rerun. **`work` may run more than once.** Keep everything that is not a write through the transaction out of it: HTTP calls, broker publishes, emails, in-memory counters, caches. Do them before the transaction starts or after it commits (the `no-external-call-in-transaction` lint rule already flags the usual ones). Events published by a failed attempt are rolled back with it, so the outbox never holds a duplicate or an orphan of an attempt. What a `publish` announces (the `onPublished` observer hook and the wake-up of the dispatcher) is held back until the attempt that committed is over, so a rerun neither reports nor wakes for events that never existed; without an isolation level both happen at publish time, as before. Do not catch a database error inside `work` and carry on: the transaction is already aborted, and relaybus can only rerun it when the error reaches it. Each attempt is a transaction of its own and is held to its own `transactionTimeout`. | Level | Rerun on serialization failure | What it prevents | | -------------------------------- | ------------------------------ | -------------------------------------------------------------------------------------------- | | none (default), `read committed` | no | Nothing beyond the database default. Without a level the call behaves exactly as before. | | `repeatable read` | yes | Non-repeatable reads and lost updates (a concurrent update of a row you updated aborts you). | | `serializable` | yes | Everything above and write skew: transactions behave as if they ran one after the other. | ## Nested calls A `@RelayBusTransaction()` method called inside a transaction joins it, as it always did. A method that asks for an `isolationLevel` can join only a transaction that relaybus opened at that same level: the level of a running transaction cannot be changed. In any other case (an outer transaction without a level, at another level, or one that your `transactionResolver` found) the call throws a `TransactionIsolationConflictError` (`RELAYBUS_TRANSACTION_ISOLATION_CONFLICT`). A method without `isolationLevel` joins any transaction, as before. A nested `bus.transaction(...)` call always opens a transaction of its own, with or without a level; only the decorator joins. ## Stores | Store | Levels | How the level is applied | | -------------------------------------------------------------------------------- | --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | PostgreSQL with `pg` (`PgExecutor`, the default for `pool` / `connectionString`) | `read committed`, `repeatable read`, `serializable` | `BEGIN ISOLATION LEVEL ...` in the same round trip that sets the transaction budget: an isolated transaction costs no statement more than a plain one. | | PostgreSQL through TypeORM | the same | TypeORM's own `dataSource.transaction('SERIALIZABLE', ...)`, so the level is set before TypeORM subscribers run, then the budget statement (one round trip more than a plain transaction). | | PostgreSQL through Kysely, Knex, Drizzle, Prisma or your own `PostgresExecutor` | the same | `SET TRANSACTION ISOLATION LEVEL ...` as the first statement of the transaction, then the budget statement (one round trip more than a plain transaction, only for transactions that ask for a level). PostgreSQL refuses the level once a query has run, so nothing may come before it. | | MongoDB, `InMemoryEventStore` | none | They run transactions at their own level. `isolationLevel` is rejected with a `ConfigurationError` before a transaction opens, because a level the database cannot give would be a silent lie. | A serialization failure is recognized through every executor: the SQLSTATE `40001` of a `pg` error (also what Kysely and Knex rethrow), TypeORM's `QueryFailedError.driverError`, the `cause` of Drizzle's query error, and Prisma's `P2034` (write conflict or deadlock), raw-query errors that carry the SQLSTATE in `meta.code`, and driver-adapter errors. A custom `PostgresExecutor` whose library wraps the error in another shape implements the optional `isSerializationFailure(error)`. ## Choosing between a lock and SERIALIZABLE - SERIALIZABLE adds no blocking of its own: its conflict detection takes no read locks, so requests that read the same data do not queue behind each other. Ordinary write locks still apply (two requests updating one row wait, and can deadlock). Under contention it costs reruns; keep `work` short and give hot paths a higher `maxAttempts`. - A lock queues every request for that key, even when the outcomes would not conflict, and deadlocks when two locks are taken in different order. - SERIALIZABLE protects every read-then-write on the data the transaction touched without naming a lock key. In exchange PostgreSQL may abort a transaction that a perfect checker would let through (a false positive), more often when a transaction scans many rows, because it tracks reads with predicate locks that it promotes from rows to pages to tables. The rerun hides those aborts; keep the reads of such a transaction indexed and short, and raise `max_pred_locks_per_transaction` if aborts stay frequent. --- Source: https://berkcansavur.github.io/relaybus/recipes/prisma # Prisma Use `@relaybus/postgres` with the Prisma executor so events are written in the same interactive transaction as your Prisma writes. ```sh pnpm add @relaybus/core @relaybus/nestjs @relaybus/postgres ``` ## Wire the store ```ts import { RelayBusModule } from '@relaybus/nestjs'; import { PostgresEventStore } from '@relaybus/postgres'; import { createPrismaExecutor } from '@relaybus/postgres/prisma'; RelayBusModule.forRootAsync({ inject: [PrismaService], useFactory: (prisma: PrismaService) => ({ store: new PostgresEventStore({ executor: createPrismaExecutor(prisma) }), source: 'orders', }), }); ``` ## Publish in a Prisma transaction ```ts await prisma.$transaction(async (tx) => { const order = await tx.order.create({ data: { total: 10 } }); await bus.publish(new OrderPlaced({ orderId: order.id }), { transaction: tx, key: order.id }); }); ``` If the transaction rolls back, the event is rolled back with it. ## Notes - Prisma interactive transactions default to a 5 second timeout. If handlers run longer, pass `createPrismaExecutor(prisma, { timeout, maxWait })`. - Move your Prisma disconnect from `onModuleDestroy` to `onApplicationShutdown`, so handlers still have their connection while the bus drains. - The Prisma adapter is typed structurally (relaybus does not depend on Prisma). It runs the full contract suite against PostgreSQL with a generated Prisma Client (7.10, and 6.19 and 5.22 in CI) on `@prisma/adapter-pg`. Prisma 5 needs `previewFeatures = ["driverAdapters"]` in its generator for the adapter. - Give the store the `pg` pool of your driver adapter too: `new PostgresEventStore({ executor: createPrismaExecutor(prisma), pool })` with `new PrismaClient({ adapter: new PrismaPg(pool) })`. The dispatcher's statements then run as prepared statements, and the store listens for the notification PostgreSQL sends when your `$transaction` commits, so the handler starts within milliseconds of the commit instead of after a fixed delay or the next poll. - Prisma 7 raw queries cannot read a `void` column; relaybus' statements never return one. See [@relaybus/postgres](https://berkcansavur.github.io/relaybus/packages/postgres) for all adapters. --- Source: https://berkcansavur.github.io/relaybus/recipes/typeorm # TypeORM Use `@relaybus/postgres` with the TypeORM executor to write events in the same transaction as your entities. ```sh pnpm add @relaybus/core @relaybus/nestjs @relaybus/postgres ``` ## Wire the store ```ts import { DataSource } from 'typeorm'; import { RelayBusModule } from '@relaybus/nestjs'; import { PostgresEventStore } from '@relaybus/postgres'; import { createTypeOrmExecutor } from '@relaybus/postgres/typeorm'; RelayBusModule.forRootAsync({ inject: [DataSource], useFactory: (dataSource: DataSource) => ({ store: new PostgresEventStore({ executor: createTypeOrmExecutor(dataSource) }), source: 'orders', }), }); ``` ## Publish in a TypeORM transaction The transaction handle is the `EntityManager` of `dataSource.transaction`: ```ts await dataSource.transaction(async (manager) => { const order = await manager.save(Order, { total: 10 }); await bus.publish(new OrderPlaced({ orderId: order.id }), { transaction: manager, key: order.id, }); }); ``` ## Avoid passing the manager around If your services already run inside a transaction opened by a library such as `@nestjs-cls/transactional`, configure a `transactionResolver` so `bus.publish(event)` joins the active transaction automatically. The resolver is called synchronously and must return `undefined` when no transaction is active. See [@relaybus/postgres](https://berkcansavur.github.io/relaybus/packages/postgres) and [@relaybus/nestjs](https://berkcansavur.github.io/relaybus/packages/nestjs). --- Source: https://berkcansavur.github.io/relaybus/recipes/mongoose # Mongoose `@relaybus/mongodb` uses MongoDB multi-document transactions, which need a **replica set or a sharded cluster** (a single-node replica set is enough for development). Mongoose exposes the same driver session. ```sh pnpm add @relaybus/core @relaybus/mongodb mongodb ``` ## Wire the store ```ts import mongoose from 'mongoose'; import { RelayBus } from '@relaybus/core'; import { MongoEventStore } from '@relaybus/mongodb'; import type { ClientSession } from 'mongodb'; const bus = new RelayBus({ store: new MongoEventStore({ client: mongoose.connection.getClient(), database: 'shop' }), }); ``` ## Publish in a Mongoose transaction ```ts const session = await mongoose.startSession(); await session.withTransaction(async () => { await Order.create([{ total: 10 }], { session }); await bus.publish(new OrderPlaced({ orderId: '1' }), { transaction: session }); }); ``` ## Things to know - A session without a started transaction is rejected with a `ConfigurationError`: MongoDB would commit each write immediately and silently break atomicity. - `session.withTransaction` may run your callback again after a transient error, so keep the callback free of side effects other than database writes. - A standalone `mongod` is rejected with a clear error. See [@relaybus/mongodb](https://berkcansavur.github.io/relaybus/packages/mongodb) for options, indexes and retention. --- Source: https://berkcansavur.github.io/relaybus/recipes/kafka-between-services # Kafka between services Service A writes an event in its database transaction. relaybus forwards it to a Kafka topic after commit. Service B stores received events once by id, then runs its handlers. The database-to-Kafka gap is covered by the outbox, and the Kafka-to-handler gap by the inbox. ```sh pnpm add @relaybus/core @relaybus/kafka @confluentinc/kafka-javascript ``` ## Service A: route committed events to a topic ```ts import { KafkaJS } from '@confluentinc/kafka-javascript'; import { RelayBus } from '@relaybus/core'; import { KafkaTransport } from '@relaybus/kafka'; const transport = new KafkaTransport({ kafka: new KafkaJS.Kafka({ kafkaJS: { clientId: 'orders', brokers: ['kafka:9092'] } }), client: 'confluent', }); const bus = new RelayBus({ store, source: 'orders', transports: [transport], routes: [{ event: OrderPlaced, transport: 'kafka', destination: 'orders.events' }], }); ``` ### Several events of a catalog to one topic When the events belong to a [catalog](https://berkcansavur.github.io/relaybus/concepts/event-catalogs), one `route` sends the ones you pick with shared settings instead of one route per event: ```ts import { produce, route } from '@relaybus/core'; const bus = new RelayBus({ store, source: 'orders', transports: [transport], routes: [ route(produce(OrderEvents, { events: ['OrderPlaced', 'OrderCancelled'] }), { transport: 'kafka', destination: 'orders.events', retry: { initialDelay: '500ms', maxDelay: '30s' }, timeout: '10s', }), ], }); ``` Events of one key still reach the topic in commit order whatever their types, and `produce` checks count the events as routed. For a topic whose format names the types differently, add `wireTypes` to the route or the catalog (see [Wire names](https://berkcansavur.github.io/relaybus/concepts/event-catalogs#wire-names-next-to-the-catalog-and-the-route)). ## Service B: subscribe and handle ```ts const transport = new KafkaTransport({ kafka: new KafkaJS.Kafka({ kafkaJS: { clientId: 'billing', brokers: ['kafka:9092'] } }), client: 'confluent', groupId: 'billing', }); const bus = new RelayBus({ store, source: 'billing', transports: [transport], subscriptions: [{ transport: 'kafka', destinations: ['orders.events'] }], }); bus.subscribe(OrderPlaced, async (event) => createInvoice(event.payload.orderId), { id: 'billing.invoice', }); ``` ## What you get - **Publish**: an idempotent producer with `acks=-1`; the route is retried until the broker acknowledges (unlimited attempts by default). - **Consume**: the offset is committed only after the event is stored in the inbox, so a crash redelivers instead of losing it, and a duplicate is stored once. - **Wire format**: the CloudEvents Kafka binding in binary mode, readable by any CloudEvents-aware tool. - **Malformed records** are copied to `.dead` by default. Handlers must still be idempotent: delivery is at-least-once. See [@relaybus/kafka](https://berkcansavur.github.io/relaybus/packages/kafka) for partitioning, ordering and using `kafkajs` instead of the Confluent client. --- Source: https://berkcansavur.github.io/relaybus/recipes/rabbitmq # RabbitMQ The RabbitMQ transport publishes outbox events with broker confirms and consumes them with manual acknowledgements, so an event is never lost between services: it is either delivered, still waiting in the outbox, or in a dead-letter queue. ```sh pnpm add @relaybus/core @relaybus/rabbitmq amqplib ``` RabbitMQ 3.13 or 4.x is required (quorum queues). ```ts import { RelayBus } from '@relaybus/core'; import { RabbitMqTransport } from '@relaybus/rabbitmq'; const rabbit = new RabbitMqTransport({ name: 'rabbit', url: process.env.AMQP_URL, exchange: 'events', consumerGroup: 'billing', // queue: relaybus.billing }); const bus = new RelayBus({ store, source: 'billing', transports: [rabbit], routes: [{ event: OrderPaid, transport: 'rabbit', destination: 'billing.order-paid' }], subscriptions: [{ transport: 'rabbit', destinations: ['orders.*'] }], }); ``` Events of a [catalog](https://berkcansavur.github.io/relaybus/concepts/event-catalogs) that share a routing key need one `route(produce(OrderEvents, { events: [...] }), { transport: 'rabbit', destination: 'billing.events' })` instead of a route per event; its optional `wireTypes` name the types for a `jsonEnvelope` codec. ## What you get - `publish` resolves only after the broker confirmed the message; otherwise the outbox keeps the delivery and retries. - Messages are acknowledged after the inbox write. A crash before that redelivers; the inbox drops the duplicate. - Failing handlers are retried through delay-tiered retry queues (about 1 hour by default) so a short database outage dead-letters nothing. Poison messages go to `.dead`. - Reconnects with capped, jittered backoff. Order is not preserved by default. Set `ordered: true` for per-queue ordering at the cost of throughput. Publishing uses the AMQP `mandatory` flag, so start consumers first or declare queues in advance. Details: [@relaybus/rabbitmq](https://berkcansavur.github.io/relaybus/packages/rabbitmq). --- Source: https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services # Interoperating with non-relaybus services relaybus only needs to run on one side of a topic or queue. The `codec` option of the Kafka and RabbitMQ transports sets the wire format in both directions: how events are written when relaybus publishes, and how messages are read when it consumes. Messages written by other services get the same guarantees as relaybus's own: stored once by id, handled once, malformed input dead-lettered while the consumer keeps running. | Other side writes or reads | `codec` | | ------------------------------------------------ | --------------------------------------------------- | | CloudEvents binary mode, or any client | `cloudEventsBinary` (default, nothing to configure) | | CloudEvents structured mode (`cloudevents+json`) | `cloudEventsStructured` | | Its own JSON envelope around the payload | `jsonEnvelope({ type, id, payload, ... })` | | The bare JSON payload, metadata in headers | `jsonEnvelope({ type, id, payload: 'whole-body' })` | | Avro, Protobuf, anything else | `{ encode, decode }`, your own codec | ```ts import { KafkaTransport, cloudEventsStructured } from '@relaybus/kafka'; new KafkaTransport({ kafka, groupId: 'billing', codec: cloudEventsStructured }); ``` The built-ins have the same names in `@relaybus/kafka` and `@relaybus/rabbitmq`. One transport is one subscription, so a service that speaks two formats uses two transports (each with its own `name`), or one codec whose `decode` switches on `record.topic` (Kafka) or `message.fields.routingKey` (RabbitMQ). ## CloudEvents binary mode, the default The payload is the JSON body, UTF-8. The attributes travel as headers. | Attribute | Kafka header | RabbitMQ header (and property) | | -------------------- | ------------------------------------------------------- | --------------------------------------------- | | spec version | `ce_specversion: 1.0` | `cloudEvents:specversion: 1.0` | | event id | `ce_id` | `cloudEvents:id` (`messageId`) | | event type | `ce_type` | `cloudEvents:type` (`type`) | | source service | `ce_source` | `cloudEvents:source` (`appId`) | | time (RFC 3339, UTC) | `ce_time` | `cloudEvents:time` (`timestamp`, seconds) | | ordering key | `ce_subject` and the record key | `cloudEvents:subject` | | content type | `content-type: application/json` | `cloudEvents:datacontenttype` (`contentType`) | | correlation id | `ce_correlationid` | `cloudEvents:correlationid` (`correlationId`) | | causation id | `ce_causationid` | `cloudEvents:causationid` | | your event headers | as is (`x-relaybus-h-` prefix only on a name collision) | `relaybus:` | Reading is strict: anything that is not binary mode is malformed. Reading it without relaybus: - **Plain kafkajs, Go, Python**: `JSON.parse(record.value)`, type and id from the `ce_type` and `ce_id` headers. - **Java**: `io.cloudevents:cloudevents-kafka` `CloudEventDeserializer` reads the record as a `CloudEvent`. Spring Cloud Function and Spring Cloud Stream recognise the `ce_` headers. - **Go**: `cloudevents/sdk-go` with `protocol/kafka_sarama`. - **RabbitMQ (any AMQP 0-9-1 client, Spring AMQP)**: bind a queue to the exchange with the routing key, parse the body, read `messageId` and `type` from the properties or the `cloudEvents:*` headers. ```java // Spring Kafka consumer of a relaybus topic, no relaybus on this side. @KafkaListener(topics = "orders.events") void onOrder(@Payload String json, @Header("ce_type") String type, @Header("ce_id") String id) { if (processedIds.add(id) && type.equals("order.placed.v1")) { handle(objectMapper.readValue(json, OrderPlaced.class)); } } ``` ## CloudEvents structured mode CloudEvents SDKs often send one JSON document with the attributes and `data`, with `content-type: application/cloudevents+json` (Kafka header, RabbitMQ `contentType` property): ```json { "specversion": "1.0", "id": "ce-5", "type": "order.placed.v1", "source": "//go/orders", "time": "2026-05-01T10:00:00Z", "subject": "order-5", "datacontenttype": "application/json", "data": { "orderId": "5" } } ``` `codec: cloudEventsStructured` writes this document (the key as `subject`, correlation and causation ids as the `correlationid` and `causationid` extension attributes, event headers as message headers) and reads both structured and binary mode, so a topic can move from one mode to the other while both are in flight. `data` must be JSON (`data_base64` is refused); string extension attributes such as `traceparent` become event headers. On RabbitMQ the AMQP properties are those of binary mode, and event header names starting with `x-death`, `x-first-death-`, `x-last-death-` or `x-relaybus-` are refused, because readers drop them. ## A JSON envelope of your own `jsonEnvelope(mapping)` declares where each value lives: a dotted body path (`{ body: 'meta.correlationId' }`, a whole number indexes an array: `'messageType.0'`) or a header (`{ header: 'event-type' }`). The codec writes and reads the same mapping, so relaybus and the other services can both produce and consume. | Field | Required | Where | When not mapped | | --------------- | -------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------- | | `type` | yes | body, header, or `{ fixed: 'order.placed.v1' }` | | | `id` | yes | body, header, or the transport's own id (below) | | | `payload` | yes | body path, or `'whole-body'` | | | `key` | no | body or header | Kafka: the record key (written unless `recordKey` says otherwise). RabbitMQ: none. | | `source` | no | body or header | Kafka: the topic. RabbitMQ: the `appId` property (always written), else the routing key. | | `occurredAt` | no | body or header, RFC 3339 (or `format`, below) | Kafka: the record timestamp (written from `occurredAt`). RabbitMQ: `timestamp`, whole seconds. | | `correlationId` | no | body or header | null | | `causationId` | no | body or header | null | | `headers` | no | body path of an object, or `'message-headers'` | `'message-headers'`: every message header no other field maps | | `contentType` | no | written as the content type | `application/json` | | `recordKey` | no | Kafka: `'event-key'`, `'none'` or `(event) => key` | `'event-key'`; `contentTypeHeader: false` also drops the `content-type` header | | `extra` | no | `{ body: { path: value }, headers: { name: value } }` | nothing extra; values are constants or `(event) => value`, written only, never read | | `bodyOrder` | no | top-level body field names | fields in the order they are written; set it when the other side compares bytes | The types make an incomplete mapping a compile error: `type`, `id` and `payload` are required, and with `payload: 'whole-body'` every other field must be a header, because the body is the payload. The id strategy is always explicit, because it is what the inbox deduplicates on: | Transport id | Written | Read | | -------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | Kafka `'record-position'` | a `relaybus-event-id` header | that header when present (records relaybus wrote), else `kafka:::`, stable for every redelivery of a record | | RabbitMQ `'message-id-property'` | the `messageId` property | the `messageId` property, kept on redeliveries; a message without one is malformed | | `{ body }` or `{ header }` | the event id at that location | the value there; a message without one is malformed | A producer that may send one event twice writes it at two Kafka offsets, so only a producer id deduplicates that; prefer `{ body }` or `{ header }` when the other side sets an id. relaybus itself writes the event id in `relaybus-event-id` under `'record-position'`, so its own outbox retries stay deduplicated. A value that is neither mapped nor carried by the transport (see the table) does not survive the trip: on RabbitMQ an unmapped key reads as none and an unmapped `occurredAt` keeps whole seconds. Map every value both sides need. `occurredAt: { body: 'timestamp', format: 'epoch-millis' }` writes whole epoch milliseconds (a number in the body, decimal text in a header) and reads them back; `'epoch-seconds'` drops the milliseconds. `extra` writes what in-house formats carry besides the event: a constant `provider: 'orders-api'`, a second copy of the time (`createdDate: (event) => event.occurredAt.toISOString()`) or a schema-version header. Extra values are never read, extra headers are left out of the event headers on read, and an extra value at a mapped location is a configuration error. A payload-only mapping takes extra headers only. On write, event headers that travel as message headers keep their names, so a name the mapping uses for another field is refused with a non-retryable error (`UnencodableEventError` on Kafka, `RabbitMqUnencodableEventError` on RabbitMQ), as is `content-type` on Kafka. A `{ fixed }` type refuses events of any other type the same way: such a route carries one type only. On read, a body field holding a number is read as its decimal text when it is a safe integer, so numeric ids work. ### A body envelope, as service monorepos use it ```json { "eventName": "order.placed.v1", "messageId": "0190a000-0000-7000-8000-0000000000aa", "data": { "orderId": "42" }, "meta": { "correlationId": "flow-1", "occurredAt": "2026-05-01T10:00:00.000Z", "headers": {} } } ``` ```ts import { KafkaTransport, jsonEnvelope } from '@relaybus/kafka'; const transport = new KafkaTransport({ kafka, groupId: 'billing', codec: jsonEnvelope({ type: { body: 'eventName' }, id: { body: 'messageId' }, payload: { body: 'data' }, correlationId: { body: 'meta.correlationId' }, causationId: { body: 'meta.causationId' }, occurredAt: { body: 'meta.occurredAt' }, headers: { body: 'meta.headers' }, }), }); ``` ### Schema validation Payloads read by any codec go through the event's schema exactly like relaybus's own: the handler of an event defined with `defineEvent(type, { schema })` gets the validated value, and a payload that fails the schema makes the delivery dead (`EventValidationError`, code `RELAYBUS_EVENT_VALIDATION`, visible through `onDeliveryDead` and the store's dead deliveries) while the consumer keeps running. Any Standard Schema library works, for example zod: ```ts import { z } from 'zod'; import { defineEvent } from '@relaybus/core'; export class OrderPlaced extends defineEvent('order.placed.v1', { schema: z.object({ orderId: z.string(), total: z.number() }), }) {} // The `data` of { "eventName": "order.placed.v1", "messageId": ..., "data": { "orderId": "42", "total": 12.5 } } // reaches the handler only when it matches the schema. bus.subscribe(OrderPlaced, async (event) => charge(event.payload.orderId, event.payload.total), { id: 'billing.charge', }); ``` Publishing validates before the codec runs, so an invalid payload is rejected by `publish` and never written. ### MassTransit MassTransit's JSON envelope carries the type as a list of message URNs and the payload under `message`, with content type `application/vnd.masstransit+json`. Subscribe to the URN as the event type (`urn:message:Orders.Contracts:OrderPlaced` is a valid relaybus type), or map it in your own codec: ```ts import { RabbitMqTransport, jsonEnvelope } from '@relaybus/rabbitmq'; const transport = new RabbitMqTransport({ url, consumerGroup: 'billing', codec: jsonEnvelope({ type: { body: 'messageType.0' }, id: { body: 'messageId' }, payload: { body: 'message' }, correlationId: { body: 'correlationId' }, causationId: { body: 'initiatorId' }, occurredAt: { body: 'sentTime' }, headers: { body: 'headers' }, contentType: 'application/vnd.masstransit+json', }), }); ``` relaybus writes only the mapped fields (`messageType` as a one-element list); `conversationId`, `sourceAddress`, `host` and the other MassTransit fields are not written and ignored on read. MassTransit publishes to one fanout exchange per message type and relaybus to one topic exchange: connect them with an exchange-to-exchange binding in your broker setup. ### The bare JSON payload A plain kafkajs, Spring `JsonSerializer` or Go producer that writes only the payload, with the type in a header: ```ts import { KafkaTransport, jsonEnvelope } from '@relaybus/kafka'; const transport = new KafkaTransport({ kafka, groupId: 'billing', codec: jsonEnvelope({ type: { header: 'event-type' }, // or { fixed: 'order.placed.v1' } for a one-type topic id: 'record-position', // or { header: 'message-id' } when the producer sets one payload: 'whole-body', }), }); ``` On RabbitMQ, use `id: 'message-id-property'` (Spring AMQP sets `messageId` with `setCreateMessageIds(true)`). The remaining message headers become event headers. ## Your own codec ```ts import type { KafkaCodec } from '@relaybus/kafka'; const legacyOrders: KafkaCodec = { encode: (event) => ({ key: event.key, value: encodeLegacyOrder(event.payload), headers: {} }), decode: (record) => { const order = decodeLegacyOrder(record.value); // synchronous and deterministic return { id: order.reference, type: 'order.placed.v1', payload: order }; }, }; ``` `decode` may leave out `source`, `key`, `occurredAt`, `headers`, `correlationId` and `causationId`; the transport fills them as in the table above. Both functions must be synchronous and free of side effects: `decode` runs again on every redelivery and must return the same id. On RabbitMQ, `encode` returns `{ content, options }` and should set `options.messageId`, which the transport uses to match unroutable returns. What `encode` throws fails the publish, and the outbox retries it unless it is a `NonRetryableError`. ### A legacy flat envelope without keys or headers Some topics carry many event types as one flat JSON object, written without a record key and without headers, with the event headers spread over the top level: ```json { "id": "evt-1", "requestId": "req-1", "provider": "billing-api", "timestamp": 1792936800123, "createdDate": "2026-10-25T14:00:00.123Z", "type": "order.created", "data": { "orderId": 1 }, "userId": "u-7" } ``` ```ts codec: jsonEnvelope({ type: { body: 'type' }, id: { body: 'id', whenMissing: 'record-position' }, // older records have no id payload: { body: 'data' }, source: { body: 'provider' }, occurredAt: { body: 'timestamp', format: 'epoch-millis' }, correlationId: { body: 'requestId' }, headers: 'top-level-fields', // event headers spread over the top level, read back from it extra: { body: { createdDate: (event) => event.occurredAt.toISOString() } }, bodyOrder: ['id', 'requestId', 'provider', 'timestamp', 'createdDate', 'type', 'data'], recordKey: 'none', contentTypeHeader: false, }), ``` `headers: 'top-level-fields'` reads every top-level text field that no other value uses as an event header. Other unknown fields (numbers, booleans, null, objects, arrays) are left out unless you keep them with `otherFields`. `bodyOrder` orders every top-level field relaybus writes, the spread headers and kept fields included; fields it does not list follow in the order they are written. To keep the unknown fields that are not text, name a payload field for them: ```ts codec: jsonEnvelope({ /* ...as above */ otherFields: { payloadField: 'wire' } }), ``` A record `{ ..., "data": { "orderId": 2 }, "userId": "u-9", "attempt": 2, "meta": { "v": 1 } }` then reads as the payload `{ "orderId": 2, "wire": { "attempt": 2, "meta": { "v": 1 } } }` with every JSON type unchanged and in body order, so the event's schema validates them with the rest of the payload. An object payload always gets the field, `{}` when the record has nothing to keep. Writing spreads `wire` back over the top level after the other fields, so the values survive a read and a write; to get the captured bytes back, list the fields in `bodyOrder` in their wire order (relaybus writes JSON without spaces). A payload that is not an object while there are fields to keep, or that already has the field, is invalid. Without `otherFields` nothing changes. ### Type names the wire spells differently Event types are 1 to 255 letters, digits, `.`, `_`, `:`, `/` or `-`, starting with a letter or digit, so snake_case names such as `transfer_auto_dispatched` work as they are. When the wire names types in a way that is not an event type, or you want relaybus names of your own, map them per route with `wireTypes` instead of a `fixed` type per transport: ```ts codec: jsonEnvelope({ type: { body: 'type' }, id: { body: 'id' }, payload: { body: 'data' }, wireTypes: { 'transfer.auto-dispatched.v1': 'Transfer Auto Dispatched', 'transfer.cancelled.v1': 'transfer_cancelled', }, }), ``` A listed type is written under its wire name and read back from it; any other type travels as itself. The same names can be declared next to the events instead: `wireTypes` of `defineCatalog` (by catalog event name) and of `route(produce(Catalog, { events }), { ... })` reach the codec of the route's transport when the bus is created and write the same bytes, so a catalog shared between services carries its wire names with it; see [Wire names](https://berkcansavur.github.io/relaybus/concepts/event-catalogs#wire-names-next-to-the-catalog-and-the-route). Wire names must be unique, an unlisted event type that equals another type's wire name is refused on write, and `wireTypes` cannot be combined with a `fixed` type. A record without a key reads as an event without a key. relaybus still keeps the outbox order of one event key, but keyless records are spread over partitions by the producer, so consumers see events of one key in order only when the key reaches the wire. ## Prove the format with captured messages Before relaybus joins a topic or queue other services already use, capture a few real messages and check, in a unit test, that relaybus writes exactly those messages and reads back exactly those events. `@relaybus/testing` registers one test per captured message: ```ts import { kafkaCodecContract } from '@relaybus/testing'; import { decodeKafkaRecord, jsonEnvelope } from '@relaybus/kafka'; const codec = jsonEnvelope({ type: { header: 'x-event-type' }, id: { header: 'x-message-id' }, payload: { body: 'payload' }, source: { body: 'meta.producer' }, occurredAt: { body: 'meta.sentAt', format: 'epoch-millis' }, extra: { body: { 'meta.schema': 'orders/v1' }, headers: { 'x-schema-version': '1' } }, bodyOrder: ['meta', 'payload'], }); kafkaCodecContract('orders format', { encode: codec.encode, decode: (record) => decodeKafkaRecord(codec, record), compare: 'bytes', // or 'json': same JSON regardless of key order and spacing ignoreHeaders: ['content-type'], // headers relaybus adds that the other consumers ignore cases: [ { name: 'order placed', envelope: { id: 'm-1', type: 'order.placed.v1', key: 'order-1' /* ... the whole event */ }, record: { topic: 'orders', key: 'order-1', headers: { 'x-event-type': 'order.placed.v1', 'x-message-id': 'm-1', 'x-schema-version': '1', }, value: '{"meta":{...},"payload":{...}}', }, }, ], }); ``` Each case checks both directions: `encode(envelope)` gives the captured key, value and headers (and the timestamp, when the codec sets one), and decoding the captured record gives `envelope`, filled in the way the transport fills it (`decodeKafkaRecord`). When the wire does not carry all of the event, for example a record without a key, give the event decoding yields as `decoded`; `only: 'decode'` checks a message only other producers write (one without an id, say), `only: 'encode'` one only other consumers read. `encode` may also be a transport's publish captured by an injected producer (`new KafkaTransport({ producer: recorder, codec })`, see [Reusing the application's producer](https://berkcansavur.github.io/relaybus/packages/kafka#reusing-the-applications-producer)), so the test covers what the transport sends. `rabbitMqCodecContract` does the same for RabbitMQ with `decodeRabbitMqMessage`: the captured message has `properties` (as publish options name them, such as `persistent`), `headers` and `body`; `ignoreProperties` leaves out a per-send `timestamp`. The routing key is the route's destination, not part of the codec. `kafkaCodecMismatches` and `rabbitMqCodecMismatches` return the differences as lines. Compared byte for byte (the default), the decoded payload must also keep the key order of the expected one. The contract runs under any test framework. `@relaybus/testing` registers with Vitest; `@relaybus/testing/jest` exports the same `kafkaCodecContract` and `rabbitMqCodecContract` for Jest (it uses Jest's `describe` and `it` globals and needs no Vitest); `@relaybus/testing/codec-contract` imports no test framework at all: `assertKafkaCodecCase(options, testCase)` throws a `CodecContractError` listing the mismatches, and `registerKafkaCodecContract({ describe, it }, { name, options })` registers one test per case with the functions of Mocha, `node:test` or any other runner (RabbitMQ: `assertRabbitMqCodecCase`, `registerRabbitMqCodecContract`). ```ts // Jest import { kafkaCodecContract } from '@relaybus/testing/jest'; ``` ## Malformed input Under every codec, a record or message that cannot become an event (not JSON, no type, an invalid type, a missing mapped id, a missing payload, an exception in your `decode`) goes to the transport's invalid-message handling: Kafka dead-letters it to `.dead` with the reason in the `relaybus-invalid-reason` header (or applies your `onInvalidMessage` policy), RabbitMQ dead-letters it to `.dead`. The consumer never crashes and moves on. Records behind it keep their per-key order. ## Migrating from `decode` `decode` still works and is deprecated: it reads with your function and writes CloudEvents binary mode. `decode: fn` is `codec: { encode: encodeEnvelope, decode: fn }`; giving both options is a configuration error. `decodeCloudEvent` and `plainJsonDecoder` remain available for such a codec, for example to read plain JSON and CloudEvents from one topic while writing binary mode: ```ts import { encodeEnvelope, fromHeader, plainJsonDecoder } from '@relaybus/kafka'; codec: { encode: encodeEnvelope, decode: plainJsonDecoder({ type: fromHeader('event-type') }) } ``` --- Source: https://berkcansavur.github.io/relaybus/recipes/testing-with-fakeclock # Testing with FakeClock Retries and delays are scheduled in the future. `FakeClock` from `@relaybus/testing` lets a test advance time instead of sleeping. ```sh pnpm add -D @relaybus/testing vitest ``` ## NestJS Use the in-memory store and turn the dispatcher off so the test controls when deliveries run: ```ts import { Test } from '@nestjs/testing'; import { InMemoryEventStore, RelayBus } from '@relaybus/core'; import { RelayBusModule } from '@relaybus/nestjs'; import { FakeClock } from '@relaybus/testing'; const clock = new FakeClock('2026-01-01T00:00:00Z'); const moduleRef = await Test.createTestingModule({ imports: [ RelayBusModule.forRoot({ store: new InMemoryEventStore(), dispatcher: { enabled: false }, clock, }), AppFeatureModule, ], }).compile(); await moduleRef.init(); await moduleRef.get(OrderService).place('o-1'); await moduleRef.get(RelayBus).drain(); // runs every due delivery clock.advance(5_000); // make retries due await moduleRef.get(RelayBus).drain(); ``` - `bus.runOnce()` runs a single batch; `bus.drain()` loops until nothing is due. - `bus.listDeliveries({ status: 'dead' })` shows handlers that exhausted their retries. - esbuild does not emit decorator metadata, so use explicit `@Inject(Token)` in test classes under Vitest. ## Testing your own store Run `eventStoreContract` from `@relaybus/testing` against a real database to prove a custom `EventStore` behaves like the built-in ones. See [@relaybus/testing](https://berkcansavur.github.io/relaybus/packages/testing). --- Source: https://berkcansavur.github.io/relaybus/comparison # relaybus vs alternatives relaybus is an event bus built on the transactional outbox. The tools below solve neighbouring problems and are often the right choice. This page states what each one documents; verify details in their documentation before deciding. | | What it is | Survives a restart | Written in your database transaction | Choose it when | | -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | **relaybus** | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries | Yes, events live in your database | Yes | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys | | [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2` | No persistence is documented | No | Simple in-process notifications where losing an event on restart is acceptable | | [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs) | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory | No built-in persistence | No | You want CQRS structure; it can sit on top of an outbox | | [`@nestjs/outbox`](https://github.com/nestjs/outbox) | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores | Yes | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing | | [BullMQ](https://docs.bullmq.io/) | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case | Yes, in Redis | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity | | [pg-boss](https://github.com/timgit/pg-boss) | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction | Yes, in PostgreSQL | Yes, through its database adapters | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports | | Kafka client directly | A distributed log; the idempotent producer and transactions give exactly-once within Kafka | Yes, in Kafka | No, producing to Kafka is a second write | You consume and produce only inside Kafka; add an outbox when a database write must be announced | This table summarises the vendors' own documentation as of October 2026; check their docs for current details. Full comparison and sources: . ## Notes on each alternative **`@nestjs/event-emitter`.** Built on `eventemitter2`. Its documentation describes an observer implementation and mentions no persistence or delivery guarantee. Errors thrown by a listener are logged by default (`suppressErrors`). It is the simplest option when events are in-process notifications. **`@nestjs/cqrs`.** Provides CommandBus, QueryBus, EventBus and sagas. Its documentation states that sagas and event handlers run in memory. It does not describe an outbox. Nothing prevents you from using it with relaybus as the durable transport underneath. **`@nestjs/outbox`.** A transactional outbox module published under the NestJS organisation. Its repository describes PostgreSQL and MySQL stores, a relay with leases, retries and a dead-letter table, and an inbox that deduplicates redeliveries. If you need MySQL, evaluate it. relaybus additionally provides a MongoDB store, Kafka and RabbitMQ transports with publisher confirms and offset-after-inbox semantics, ordering keys and a contract-tested store interface. **BullMQ.** A queue library on Redis with retries, delayed jobs and flows. Its documentation says it aims for exactly-once delivery and delivers at least once in the worst case. Because jobs live in Redis, adding a job cannot be part of your SQL or MongoDB transaction. It is a good fit when you already run Redis and need background job features. **pg-boss.** A PostgreSQL job queue using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub. Its documentation says jobs can be created inside an existing transaction through adapters for Drizzle, Knex, Kysely and Prisma. It models jobs for queues; relaybus models events with one delivery per subscribed handler, per-key ordering and broker routes. For cron-style schedules, prefer a job queue. **Kafka directly.** Kafka's idempotent producer and transactions give exactly-once semantics within Kafka. Its own design documentation notes that exactly-once with an external system needs cooperation from that system. Producing to Kafka and writing your database are two writes; an outbox makes them one. ## Choosing quickly | If you need | Consider | | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------- | | In-process notifications, loss on restart acceptable | `@nestjs/event-emitter` | | CQRS structure (commands, queries, sagas) | `@nestjs/cqrs`, optionally with relaybus for durable events | | Atomic publish with PostgreSQL or MongoDB, per-handler retries, ordering keys, Kafka or RabbitMQ | relaybus | | Transactional outbox on MySQL | `@nestjs/outbox` | | Background jobs and rate limiting on Redis | BullMQ | | A PostgreSQL job queue with scheduling | pg-boss | | Streaming processing entirely inside Kafka | Kafka clients and Kafka transactions | ## Sources - - - - - - Comparison checked on 18 October 2026. --- Source: https://berkcansavur.github.io/relaybus/tuning # Tuning relaybus for your workload The defaults are chosen to be fast for the common case: an API that publishes inside its own transactions and runs the dispatcher in the same process, against PostgreSQL directly. Change a setting only when your infrastructure or traffic differs from that, and change the one that matches the difference. ## The settings that matter **PostgreSQL store** (`new PostgresEventStore({ ... })`) | Option | Default | Trade-off | | -------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `preparedStatements` | `true` | The statements relaybus repeats (publish, claim, complete, renew) are parsed and planned once per connection instead of on every call. Turn off only behind a pooler that cannot keep prepared statements (PgBouncer before 1.21 in transaction mode). | | `wakeUp` | `'notify'` | `'notify'`: publishers send `NOTIFY` (at most one per 10 ms per store) and dispatchers `LISTEN`, so other processes run a new event within milliseconds. `'poll'`: no `NOTIFY` and no listening connection; other processes find new events at their next poll. | | `completionHistory` | `false` | `true` keeps a row per completed delivery for audits and `listDeliveries({ status: 'completed' })`, at one more heap and index write per delivery. | | `purgeBatchSize` | `1000` | Rows deleted per purge transaction; smaller batches hold row locks for less time. | **Dispatcher** (`new RelayBus({ dispatcher: { ... } })`, or the `dispatcher` option of `RelayBusModule.forRoot`) | Option | Default | Trade-off | | ----------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | `enabled` | `true` | `false` on processes that only publish: they then claim nothing and open no listening connection. | | `concurrency` | `10` | Handlers running at once per process. Raise it for I/O-bound handlers; each running handler may need a connection for its completion. | | `batchSize` | concurrency | Deliveries claimed per statement. Larger batches mean fewer claims per delivery. | | `pollInterval` | `'1s'` | How often an idle dispatcher looks for work it was not woken for: retries that became due, and every new event when nothing wakes it. | | `leaseDuration` | `'30s'` | How long a crashed worker's deliveries wait before another worker takes them over. Must exceed your slowest handler's time between renewals. | | `completionBatchWindow` | `'2ms'` | Completions finishing within this window are written with one statement. `0` writes every completion on its own. | Size the `pg` pool (`max`) for the publishing transactions you run at once plus the dispatcher's `concurrency`. With `wakeUp: 'notify'` each dispatching process opens one more connection for `LISTEN`, outside the pool. ## Single-connection API A service that publishes one event per request inside its own transaction and handles the events itself. The defaults fit: one statement per publish, prepared once per connection, and the publishing process wakes its own dispatcher right after the commit. ```ts import { Pool } from 'pg'; import { RelayBus } from '@relaybus/core'; import { PostgresEventStore } from '@relaybus/postgres'; const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 10 }); const bus = new RelayBus({ store: new PostgresEventStore({ pool }), source: 'orders', }); await bus.transaction(async (transaction) => { await transaction.query('INSERT INTO orders (id, total) VALUES ($1, $2)', [orderId, total]); await bus.publish(new OrderPlaced({ orderId, total }), { transaction }); }); ``` Publish several events of one transaction with one `publishAll` call: it is still one statement. ## High-traffic API with many connections Many API instances, each with a large pool and many requests in flight, publishing far more than they handle. Let the API processes only publish and leave the handling to a worker fleet (next section). Keep `wakeUp: 'notify'`: each store sends at most one notification per 10 ms, so concurrent publishers do not queue behind PostgreSQL's notification lock, and workers still start within milliseconds. ```ts const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 40 }); const bus = new RelayBus({ store: new PostgresEventStore({ pool }), source: 'checkout-api', dispatcher: { enabled: false }, }); ``` Use one store per process: the notification interval is kept per store instance. ## Worker fleet Processes that only run handlers, several of them against one database. Claim in larger batches and run more handlers at once; workers share the backlog through lane leases, so adding a worker adds throughput until the database is the limit. ```ts const pool = new Pool({ connectionString: process.env.DATABASE_URL, max: 20 }); const bus = new RelayBus({ store: new PostgresEventStore({ pool }), source: 'billing-worker', setupStore: false, // the API processes or your migrations create the tables dispatcher: { concurrency: 50, batchSize: 200, leaseDuration: '60s', // longer than the slowest handler between renewals }, }); ``` Raise `concurrency` for handlers that mostly wait on I/O; keep it near the pool size for handlers that write to the database in the delivery's transaction (`transactional: true`). Keep `pollInterval` at `'1s'`: notifications start new work, the poll only picks up retries and anything a lost notification missed. ## The transaction budget and who owns the pool `bus.transaction` holds every transaction to `transactionTimeout` (default `'5s'`). Where the database applies that budget depends on who created the pool: | Pool | Where the budget lives | Cost | | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | Created by the store (`connectionString` or `poolConfig`) | Each connection keeps the budget as a session setting. The first `bus.transaction` on a connection sets it with `BEGIN`; later ones send nothing extra. | A publish costs what it did without a budget: `BEGIN`, the insert, `COMMIT`. | | Passed in by you (`pool`, or an executor over Prisma, Knex, ...) | The budget is set for the transaction only and is gone when it ends. Your pool's session settings are never changed. | Nothing outside the transaction is affected, and the budget is set again with every transaction. | The trade-off of the owned pool: statements that run **outside** a transaction on that connection (claiming, completing and renewing deliveries) run under the budget the connection last received. With the default `'5s'` that is a `statement_timeout` of 5 seconds on those statements as well, and a raised budget (say `'30s'`) raises it for them. A transaction without a budget puts the session defaults back in its own `BEGIN`. If you need those statements to follow your own database settings, or want the budget to stay strictly per transaction, pass a pool you created. With `preparedStatements: false`, which you set behind a pooler in transaction mode, the budget stays per transaction on every pool, because session settings would reach other clients. ## PgBouncer in transaction mode A pooler in transaction mode hands every transaction a different server session. Two things need a session: - `LISTEN`: notifications are never delivered through it. Use `wakeUp: 'poll'` and shorten `pollInterval` to the latency you need, or give the dispatchers a direct connection (session mode) instead. - Prepared statements: PgBouncer 1.21 and later keeps them per client when `max_prepared_statements` is above 0, and then the default works. With an older PgBouncer, or with that setting at 0, use `preparedStatements: false`. If you forget, the first statement that lands on another session fails once, the store stops preparing and logs `[PgExecutor](query) The server does not keep prepared statements per connection`. ```ts const pool = new Pool({ connectionString: process.env.PGBOUNCER_URL, max: 20 }); const bus = new RelayBus({ store: new PostgresEventStore({ pool, wakeUp: 'poll', preparedStatements: false, // PgBouncer < 1.21, or max_prepared_statements = 0 }), source: 'orders', dispatcher: { pollInterval: '200ms' }, }); ``` A publish made in the same process still wakes that process's dispatcher at once; `pollInterval` only bounds how late other processes see it. ## Through Prisma, TypeORM, Drizzle, Knex or Kysely The adapters run relaybus' statements through your library, which sends every statement unprepared, so the database parses and plans each publish and claim again (about 0.3 ms per publish and 1.3 ms per claim of 10 deliveries in our measurements), and a store on an adapter has no connection to `LISTEN` on. Give the store the `pg` pool your library's driver uses as well. Your transactions still go through the library; the statements the store sends outside them (the dispatcher's claims, completions and lease renewals) run on the pool as prepared statements, and the store listens for wake-ups: ```ts import { Pool } from 'pg'; import { PrismaPg } from '@prisma/adapter-pg'; import { createPrismaExecutor } from '@relaybus/postgres/prisma'; const pool = new Pool({ connectionString }); const prisma = new PrismaClient({ adapter: new PrismaPg(pool) }); const bus = new RelayBus({ store: new PostgresEventStore({ executor: createPrismaExecutor(prisma), pool }), source: 'orders', }); ``` The same works with Kysely (`new PostgresDialect({ pool })`), Drizzle (`drizzle(pool)`) and any other library on a `pg` pool; for Knex and TypeORM pass a `pg` pool on the same database. A warm dispatch batch (claim and complete) parses 2 statements without `pool` and none with it. ## Wake-up after your transaction commits A delivery runs once its transaction has committed. How soon the dispatcher of the publishing process learns about the commit depends on who owns the transaction: | Transaction | Dispatcher woken | | --------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | relaybus' own (`publish` without a transaction, `bus.receive`) | right after the commit | | yours, PostgreSQL store with `pool`/`connectionString` (or `executor` plus `pool`) and `wakeUp: 'notify'` | at the commit: the publish statement sends `pg_notify` inside your transaction and PostgreSQL delivers it when the transaction commits, however long it takes. Measured: handler start a median of under 10 ms after a commit held back 200 ms. | | yours, any other store (MongoDB, `wakeUp: 'poll'`, an executor without `pool`) | 25, 100 and 250 ms after the latest publish, then at the next poll (`pollInterval`, default 1 s). Call `bus.wake()` right after your commit to claim at once. | `bus.transaction` and `@RelayBusTransaction()` count as your transaction here. `bus.wake()` is harmless when the dispatcher is not running, so it can sit in a shared "after commit" hook of your unit of work. ## MongoDB replica set The MongoDB store needs a replica set (transactions) and has no notifications: a dispatcher is woken at once by publishes of its own process and finds the rest at its next poll. Size the client pool for publishers plus `concurrency`, and set `pollInterval` to the latency other processes may add. ```ts import { MongoClient } from 'mongodb'; import { MongoEventStore } from '@relaybus/mongodb'; const client = await new MongoClient(process.env.MONGO_URL, { maxPoolSize: 50 }).connect(); const bus = new RelayBus({ store: new MongoEventStore({ client, database: 'orders' }), source: 'orders', dispatcher: { concurrency: 20, pollInterval: '250ms' }, }); ``` Keep the default `transactionOptions` (snapshot reads, majority writes): weaker write concerns can lose an acknowledged event on failover. ## Related - [Leases and fencing](https://berkcansavur.github.io/relaybus/concepts/leases) - [Guarantees](https://berkcansavur.github.io/relaybus/guarantees) - [@relaybus/postgres](https://berkcansavur.github.io/relaybus/packages/postgres) --- Source: https://berkcansavur.github.io/relaybus/upgrading # Upgrading relaybus - [0.4.1 → 0.5.0](#upgrading-from-0-4-1-to-0-5-0) - [0.4.0 → 0.4.1](#upgrading-from-0-4-0-to-0-4-1) - [0.3 → 0.4](#upgrading-from-0-3-to-0-4) - [0.2 → 0.3](#upgrading-from-0-2-to-0-3) - [0.1 → 0.2](#upgrading-from-0-1-to-0-2) ## Upgrading from 0.4.1 to 0.5.0 ```yaml applies to: '@relaybus/* 0.4.1 -> 0.5.0' downtime: 'none' rollout: 'mixed 0.4.1 and 0.5.0 instances are safe' database: 'none' rollback: 'safe; 0.5.0 stores and sends nothing 0.4.1 cannot read' ``` **Nothing is required.** 0.5.0 adds API and changes no behaviour of existing code: routes, `jsonEnvelope` `wireTypes`, handler ids, stored deliveries and the wire format are as in 0.4.1. Optional steps: 1. **Replace per-event routes with a catalog route.** Routes of one destination written event by event can become one `route`; the handler id `relaybus.transport::` and the routed event types stay the same, so deliveries stored under the old routes are still sent. Do it in one deploy; there is nothing to migrate. ```ts // before routes: [ { event: OrderPlaced, transport: 'kafka', destination: 'orders', retry }, { event: OrderCancelled, transport: 'kafka', destination: 'orders', retry }, ], // after routes: [ route(produce(OrderEvents, { events: ['OrderPlaced', 'OrderCancelled'] }), { transport: 'kafka', destination: 'orders', retry, }), ], ``` 2. **Move wire names to the catalog.** Names from the `wireTypes` of a `jsonEnvelope` can be declared by `defineCatalog(..., { wireTypes })` or `route(..., { wireTypes })` instead. The bytes are identical; naming the same pair in both places is allowed, naming another name for the same event type fails the boot with `RELAYBUS_WIRE_TYPE_COLLISION`. 3. **Replace a lock with SERIALIZABLE.** A `pg_advisory_xact_lock` or `SELECT ... FOR UPDATE` that only guards a count-check-insert can become `@RelayBusTransaction({ isolationLevel: 'serializable' })` (or `bus.transaction(work, { isolationLevel: 'serializable' })`), which does not block and reruns the transaction after a serialization failure. The method may then run more than once, so keep calls to other services out of it. Test it under concurrency first; see [Serializable transactions](https://berkcansavur.github.io/relaybus/recipes/serializable-transactions). Nothing changes for code that does not pass `isolationLevel`. Things to know: - A custom `PostgresExecutor` keeps working: without the new optional `transactionWithIsolation` the store sends `SET TRANSACTION ISOLATION LEVEL ...` as the first statement of its `transaction`, and it recognizes SQLSTATE `40001` in `code`, `cause` and `driverError` on its own (the optional `isSerializationFailure` covers other shapes). A custom `EventStore` that does not declare `isolationLevels` rejects every `isolationLevel`, so a service that never passes one is not affected. - A custom `Transport` written outside relaybus keeps working. It only needs `declareWireTypes(wireTypes)` if you declare wire names for it; without the method, declaring them fails the boot with `RELAYBUS_WIRE_TYPES_UNSUPPORTED`. Without wire names, it is never called. - `RelayBusOptions.routes` is now `readonly RouteDefinition[]` (`RouteOptions | CatalogRoute`). Code that only passes routes compiles unchanged; code that reads `options.routes` and expects `RouteOptions` now has to narrow. - A service that only consumes a catalog with wire names has no route to declare them: give its codec `wireTypes: catalogWireTypes(OrderEvents)`. ## Upgrading from 0.4.0 to 0.4.1 ```yaml applies to: '@relaybus/* 0.4.0 -> 0.4.1' downtime: 'none; publishes and claims wait while setup() rewrites the events table once' rollout: 'mixed 0.4.0 and 0.4.1 instances are safe with the defaults' database: 'PostgreSQL: payload and headers of the events table become json (one table rewrite)' rollback: 'safe with the defaults; 0.4.0 reads and writes the json columns, no DDL' ``` **Why.** The PostgreSQL store kept `payload` and `headers` as `jsonb`, which re-orders object keys (shorter keys first, then alphabetically). An event read back from the outbox therefore serialized differently from what the producer published: JSON consumers did not notice, byte or string comparisons and consumers that depend on field order did. 0.4.1 stores both as `json`, which keeps the text the producer serialized, and hands the transport and the handlers exactly that. MongoDB and the in-memory store already kept the order. **Rows already stored keep the order `jsonb` gave them.** Only events published after the upgrade keep their producer's key order. The values of old rows are unchanged. ### With `setup()` (the default, `setupStore: true`) Nothing to do. The first 0.4.1 instance that starts converts the two columns in place. It is idempotent and guarded: it runs only while a column is still `jsonb`. The conversion rewrites the events table once under an exclusive lock, so publishes and claims wait for it. It takes about as long as copying the table; on a large table run it at a quiet time, or run the DDL below yourself first. `setup()` runs it in a transaction of its own, so it cannot deadlock with running completions, and on a schema 0.4.0 created (recognized by the marker 0.4.0 left on the lane table) that is all it does: the other schema statements, whose table locks could queue behind running claims, do not run again. 0.4.1 keeps writing the marker 0.4.0 writes and reads whether the conversion ran from the column types, so once both columns are `json` its `setup()` runs nothing and takes no lock. ### Without `setup()` (`setupStore: false`, your own migrations) Run this once, with your schema and table prefix (the defaults are `public` and `relaybus_`): ```sql ALTER TABLE "public"."relaybus_events" ALTER COLUMN headers DROP DEFAULT, ALTER COLUMN payload TYPE json USING payload::json, ALTER COLUMN headers TYPE json USING headers::json, ALTER COLUMN headers SET DEFAULT '{}'::json; ``` `createPostgresSchemaSql()` contains the same conversion guarded by a column type check, so re-applying its output is safe as well. ### During the rollout 0.4.0 and 0.4.1 instances can run side by side. Before the conversion, 0.4.1 publishes into the `jsonb` columns (PostgreSQL casts on assignment; key order is lost as before); after it, 0.4.0 publishes into the `json` columns the same way. Mixed rollouts and rollbacks are safe with the default settings: a 0.4.0 instance that starts or restarts with `setupStore: true` after the conversion finds its own schema marker, which 0.4.1 keeps, and runs no schema statement. Rolling back needs no DDL; the columns stay `json`, and a later 0.4.1 start has nothing to do. ### Also in 0.4.1 - **Prisma 5.** `@relaybus/postgres` declares its optional `@prisma/client` peer as `^5.0.0 || ^6.0.0 || ^7.0.0`. With 0.4.0 (`^6 || ^7`), npm refused to install it next to Prisma 5 (`ERESOLVE`) or installed a second Prisma Client. Nothing changes in your code; Prisma 5 needs `previewFeatures = ["driverAdapters"]` for `@prisma/adapter-pg`. - **jsonEnvelope.** `bodyOrder` now also places the event headers spread by `headers: 'top-level-fields'`; before, they always came last, even when `bodyOrder` named them. Only a mapping whose `bodyOrder` names a spread header writes different bytes, and those are the bytes it asked for; headers it does not name still come last. New options: `otherFields` keeps unknown top-level fields of every JSON type, `wireTypes` names event types differently on the wire. See [Interoperating with other services](https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services). - **Codec contract.** Compared byte for byte, the decoded payload must now also keep the expected key order, so a case whose expected payload lists keys in another order than the wire fails with a `decoded payload` line; write the expected payload in wire order or use `compare: 'json'`. Jest projects use `@relaybus/testing/jest`; `vitest` is now an optional peer. ## Upgrading from 0.3 to 0.4 ```yaml applies to: '@relaybus/* 0.3.x -> 0.4.0' downtime: 'none; a rolling deploy is supported' database: 'no migration; setup() has nothing new to apply' rollback: 'safe; 0.4.0 writes the same rows, SQL and wire format as 0.3.x' ``` **There is nothing to do.** The public API only grew, defaults are unchanged and the stored and wire formats are the same. Update the packages (keep every `@relaybus/*` on one version) and deploy: ```sh pnpm add @relaybus/core@0.4.0 @relaybus/postgres@0.4.0 ``` Everything below is optional. ### Optional: give an ORM store the pool With Prisma, Kysely or Drizzle, pass the `pg` pool your ORM's driver uses next to `executor`. Claims run as prepared statements, and an event published in your ORM transaction is dispatched right after the commit instead of at the next poll: ```ts const pool = new Pool({ connectionString }); const prisma = new PrismaClient({ adapter: new PrismaPg(pool) }); const store = new PostgresEventStore({ executor: createPrismaExecutor(prisma), pool }); ``` ### Optional: wake the dispatcher after your own commit For a store without commit notifications (MongoDB, `wakeUp: 'poll'`, an executor without `pool`): ```ts await myTransaction.commit(); bus.wake(); ``` Without it the dispatcher is woken 25, 100 and 250 ms after the latest publish and otherwise at the next poll. ### Optional: the Confluent Kafka client ```ts import { KafkaJS } from '@confluentinc/kafka-javascript'; const kafka = new KafkaJS.Kafka({ kafkaJS: { clientId: 'orders', brokers: ['kafka:9092'] } }); new KafkaTransport({ kafka, client: 'confluent', groupId: 'billing' }); ``` `kafkajs` stays the default and keeps working. The two clients differ in a few behaviours; read [Choosing a client](https://berkcansavur.github.io/relaybus/packages/kafka#choosing-a-client) first and roll it out per service. ### Optional: control the Kafka record key ```ts jsonEnvelope({ /* mapping */ recordKey: 'none' }); // or (event) => event.payload.accountId ``` The default `'event-key'` is what 0.3 wrote. A keyless record is placed by the producer, so per-key order then holds only up to the broker. ### Optional: publish through your own producer ```ts const producer = kafka.producer(); // already connected, owned by your application await producer.connect(); new KafkaTransport({ producer }); // publish-only; add kafka and groupId to consume as well ``` The transport only calls `send`; it never connects or disconnects the producer. Do not combine `producer` with `producerConfig` or `maxInFlightRequests` (a `ConfigurationError`): configure the producer itself. ### Optional: in-house formats and their tests ```ts jsonEnvelope({ type: { header: 'x-event-type' }, payload: { body: 'payload' }, occurredAt: { body: 'sentAt', format: 'epoch-millis' }, extra: { body: { schema: 'orders/v1' }, headers: { 'x-schema-version': '1' } }, }); ``` ```ts import { kafkaCodecContract } from '@relaybus/testing'; kafkaCodecContract('orders format', { encode: codec.encode, decode, cases }); ``` See [Interoperating with non-relaybus services](https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services) for legacy flat envelopes and the contract helpers. ## Upgrading from 0.2 to 0.3 ```yaml applies to: '@relaybus/* 0.2.x -> 0.3.0' downtime: 'none; a rolling deploy is supported' database: 'no migration; setup() has nothing new to apply' rollback: 'roll forward; 0.2.x does not know the new route ids of deliveries written by 0.3.0' ``` Your code keeps compiling and running: every renamed API still works under its old name. Five things behave differently and are worth checking, in this order. Coming from 0.1, upgrade to 0.2 first (it deletes completed deliveries and upgrades the schema in place), then follow this page. 1. [Update the packages](#step-1-update-the-packages) 2. [Route ids changed](#step-2-route-ids-changed) 3. [Transactions have a time budget](#step-3-transactions-have-a-time-budget) 4. [stop() returns a StopResult](#step-4-stop-returns-a-stopresult) 5. [Renames: decorators and codec](#step-5-renames-decorators-and-codec) 6. [Error codes and the Mongo error field](#step-6-error-codes) 7. [Optional new packages](#step-7-optional-new-packages) ### Step 1: update the packages Keep every `@relaybus/*` package on the same version: ```sh pnpm add @relaybus/core@0.3.0 @relaybus/nestjs@0.3.0 @relaybus/postgres@0.3.0 ``` Do the same for `mongodb`, `kafka`, `rabbitmq` and `testing` if you use them. Node.js 22.12 or newer is still required. ### Step 2: route ids changed A route sends deliveries of an event type to a transport destination. Routes to one destination used to be one handler per event type; they are now one handler per destination, so events of one key but different types reach the broker in commit order. | | 0.2.0 | 0.3.0 | | ---------------- | ------------------------------------------------- | --------------------------------- | | Route handler id | `relaybus.transport:kafka:orders:order.placed.v1` | `relaybus.transport:kafka:orders` | **Nothing has to be migrated.** Deliveries stored under a 0.2.0 id, including those written by 0.2.0 processes during a rolling deploy, are still claimed and sent by the new route. The stored rows keep their old id. #### Order during the upgrade A delivery under a 0.2.0 id and one under the new id are two queues, so one key's backlog from 0.2.0 is not ordered against events published after the upgrade. Where cross-type order matters, drain the old backlog first: 1. Stop publishing with 0.2.0 (stop the 0.2.0 publishers before the 0.3.0 ones start). 2. Wait until no delivery is left under the old ids: ```ts const old = await bus.listDeliveries({ handler: 'relaybus.transport:kafka:orders:order.placed.v1', // one query per routed event type status: 'pending', }); // continue when old is empty ``` 3. Start the 0.3.0 publishers. If order across types does not matter for your consumers, skip this and deploy normally. #### Two routes that no longer construct Routes to the same destination now have to resolve to the same `retry` and `timeout`, and an event type may be routed to a destination once; otherwise `new RelayBus` throws a `ConfigurationError`. ```ts // Before (0.2.0): accepted routes: [ { event: OrderPlaced, transport: 'kafka', destination: 'orders', retry: { attempts: 5 } }, { event: OrderCancelled, transport: 'kafka', destination: 'orders' }, ], // After (0.3.0): the same settings on both routes routes: [ { event: OrderPlaced, transport: 'kafka', destination: 'orders', retry: { attempts: 5 } }, { event: OrderCancelled, transport: 'kafka', destination: 'orders', retry: { attempts: 5 } }, ], ``` #### Dashboards and code that match ids `handlers()`, `listDeliveries({ handler })`, `HandlerContext.handler` and observer events show the new id for new deliveries. Update alerts that filter on the old id; query both ids while the 0.2.0 backlog drains. ### Step 3: transactions have a time budget `transactionTimeout` is new and defaults to `'5s'`. A `bus.transaction(...)` or `@RelayBusTransaction()` method that runs longer is rolled back before its commit, and the caller gets a `TransactionTimeoutError` (`RELAYBUS_TRANSACTION_TIMEOUT`). The database enforces the budget where it can: on PostgreSQL `statement_timeout` and, with the `pg` executor, `idle_in_transaction_session_timeout` (and `transaction_timeout` on PostgreSQL 17 or newer); on MongoDB `maxCommitTimeMS`. First look for transactions that call another service. The usual fix is to move the call out: ```ts // Before: an HTTP call holds the transaction (and its row locks) open await bus.transaction(async (tx) => { const rate = await fetch(ratesUrl).then((r) => r.json()); await orders.insert(tx, { ...order, rate }); await bus.publish(new OrderPlaced({ orderId }), { transaction: tx }); }); // After: call first, then a short transaction const rate = await fetch(ratesUrl).then((r) => r.json()); await bus.transaction(async (tx) => { await orders.insert(tx, { ...order, rate }); await bus.publish(new OrderPlaced({ orderId }), { transaction: tx }); }); ``` If a transaction legitimately needs longer (a large `publishAll`, a migration job), raise the budget for the bus: ```ts new RelayBus({ store, source: 'orders', transactionTimeout: '30s' }); // NestJS RelayBusModule.forRoot({ store, source: 'orders', transactionTimeout: '30s' }); ``` `@relaybus/eslint-plugin` ([step 7](#step-7-optional-new-packages)) finds external calls inside transactions at lint time. **Who owns the pool matters.** On a pool the store creates from `connectionString` or `poolConfig`, each connection keeps the budget as a session setting, so a publish costs no extra statement; the price is that statements outside a transaction on that connection (claims, completions, renewals) run under the budget the connection last received, `'5s'` by default. On a pool or executor you pass in, the budget lasts only for the transaction and your pool's settings are never changed. Details and when to choose which: [Tuning](https://berkcansavur.github.io/relaybus/tuning#the-transaction-budget-and-who-owns-the-pool). ### Step 4: stop() returns a StopResult `bus.stop()` is now bounded: the whole shutdown takes at most `timeout` (default `dispatcher.shutdownTimeout`, 30 s) and resolves with what happened. ```ts // Before (0.2.0): Promise, no deadline await bus.stop(); // After (0.3.0): Promise const { graceful, abandonedDeliveries, durationMs } = await bus.stop({ timeout: '25s' }); if (!graceful) logger.warn(`shutdown forced, ${abandonedDeliveries} deliveries left to be taken over`); ``` - Past the deadline, relaybus force-closes what it opened itself (a store's own pool, its `LISTEN` connection, a RabbitMQ connection) and never closes a pool or client you passed in. It never rejects because of the deadline. - Abandoned deliveries keep their lease until it expires, then another process runs them again. This is the at-least-once guarantee; keep handlers idempotent. - Align the timeout with Kubernetes: `terminationGracePeriodSeconds` should be larger than `shutdownTimeout`. With NestJS call `app.enableShutdownHooks()`; `RelayBusModule` stops the bus in `beforeApplicationShutdown`, now bounded. - Existing `await bus.stop()` calls compile and behave as before; only code that needs to know whether the shutdown was graceful has to read the result. ### Step 5: renames: decorators and codec Both old names keep working, are marked `@deprecated` and are removed in 1.0.0. Moving now means the editor shows no strikethrough and the upgrade to 1.0.0 is a no-op. #### Decorators ```ts // Before (0.2.0) import { Subscribe, EventConsumer } from '@relaybus/nestjs'; @Subscribe(OrderPlaced) async onPlaced(event: EventEnvelope) {} @EventConsumer(OrderEvents) export class BillingHandlers implements HandlersOf {} // After (0.3.0) import { RelayBusConsumer, RelayBusCatalogConsumer } from '@relaybus/nestjs'; @RelayBusConsumer(OrderPlaced) async onPlaced(event: EventEnvelope) {} @RelayBusCatalogConsumer(OrderEvents) export class BillingHandlers implements HandlersOf {} ``` | Old | New | | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | | `@Subscribe` (method) | `@RelayBusConsumer` (method, the default style) | | `@EventConsumer` (class) | `@RelayBusCatalogConsumer` (class, optional) | | Their option and metadata types | `RelayBusConsumerOptions`, `RelayBusConsumerMetadata`, `RelayBusCatalogConsumerOptions`, `RelayBusCatalogConsumerMetadata` | `@RelayBusConsumer` only compiles on a method and takes an event; `@RelayBusCatalogConsumer` only on a class and takes a catalog. Handler ids are unchanged (`ClassName.methodName` by default), so no stored delivery is orphaned. #### `decode` becomes `codec` `KafkaTransport` and `RabbitMqTransport` take one symmetric `codec` for the wire format, both for publishing and for consuming. The `decode` option reads with your function and writes the default format; it is the same as `codec: { encode: encodeEnvelope, decode }`. ```ts // Before (0.2.0): inbound only new KafkaTransport({ kafka, groupId: 'billing', decode: plainJsonDecoder({ type: fromHeader('event-type') }), }); // After (0.3.0) new KafkaTransport({ kafka, groupId: 'billing', codec: { encode: encodeEnvelope, decode: plainJsonDecoder({ type: fromHeader('event-type') }) }, }); // or one of the built-in codecs: cloudEventsBinary (default), cloudEventsStructured, jsonEnvelope({ ... }) ``` Giving both `decode` and `codec` is a configuration error. The default codec, `cloudEventsBinary`, writes exactly the bytes 0.2.0 wrote (pinned by golden tests), so services you did not upgrade keep reading your messages. See the [interoperability recipe](https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services). ### Step 6: error codes - **Every relaybus error has a `code`.** `ErrorInfo` (the `lastError` of a delivery, the `error` of observer events) gains an optional `code` for relaybus errors; other errors are unchanged. Records written before 0.3.0 have no `code`, so read it as optional. The list is on [Error codes](https://berkcansavur.github.io/relaybus/concepts/error-codes). - **Five error classes report their own name.** `InvalidKafkaMessageError`, `RabbitMqUnavailableError`, `RabbitMqPublishRejectedError`, `UnroutableMessageError` and `MalformedMessageError` used to report `name: 'RelayBusError'`. Alerts that matched that name should match the class name or, better, the `code`. - **MongoDB keeps the code.** The MongoDB store now stores `code` in the `lastError` of a delivery, as PostgreSQL always did (it stores `ErrorInfo` as JSON). Documents without `code` stay valid. - **Broker dead letters carry more headers.** Kafka adds `relaybus-error-name` and `relaybus-error-code` to records copied to `.dead`; RabbitMQ adds `x-relaybus-dead-letter-error-name` and `x-relaybus-dead-letter-error-code`. The reason headers are unchanged. - **`onFailure` has a new stage, `'backlog'`.** Add a case if you `switch` exhaustively over the stage. ### Step 7: optional new packages Neither is required, and neither changes behaviour until you add it. ```sh pnpm add -D @relaybus/eslint-plugin # lint: no external call inside a transaction, no publish outside one pnpm add @relaybus/otel @opentelemetry/api # metrics and traces ``` ```js // eslint.config.js import tseslint from 'typescript-eslint'; import relaybus from '@relaybus/eslint-plugin'; export default tseslint.config(...tseslint.configs.recommended, relaybus.configs.recommended); ``` ```ts import { relayBusOpenTelemetry } from '@relaybus/otel'; const otel = relayBusOpenTelemetry(); // global meter and tracer providers new RelayBus({ store, observers: [otel.observer], tracer: otel.tracer, backlogInterval: '30s' }); ``` See [Transactions in NestJS](https://berkcansavur.github.io/relaybus/recipes/transactions-in-nestjs) and [Observability](https://berkcansavur.github.io/relaybus/concepts/observability). ### Checklist - [ ] All `@relaybus/*` packages on 0.3.0 - [ ] Routes to one destination share `retry` and `timeout` - [ ] Old route backlog drained, if cross-type order matters - [ ] No transaction waits on a slow external call (or `transactionTimeout` raised on purpose) - [ ] On a pool the store owns, you accept that statements outside transactions share the transaction budget (or pass your own pool) - [ ] `stop()` callers handle `StopResult`; grace period larger than `shutdownTimeout` - [ ] Dashboards read both route ids during the drain ## Upgrading from 0.1 to 0.2 ```yaml applies to: '@relaybus/* 0.1.0 -> 0.2.0' downtime: 'a short stop of every 0.1.0 worker (the upgraded schema is not compatible with 0.1.0 code)' rollback: 'restore from a backup; there is no down migration' ``` 0.2.0 changes how deliveries are stored ([why](https://berkcansavur.github.io/relaybus/performance/write-amplification)). Your code keeps working: `publish`, `@Subscribe`, handlers, retries and dead letters have the same API. What changes is the database layout, which `setup()` upgrades in place, and one visible behaviour: completed deliveries are deleted. ### What changes for you | 0.1.0 | 0.2.0 | Action | | -------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | | A completed delivery stays as a `completed` row until retention purges it. | It is **deleted** when it completes. `listDeliveries({ status: 'completed' })` is empty and `statistics().completed` is 0. | If you read completed deliveries, set `completionHistory: true`. | | Each delivery has lease columns and a `processing` status. | Workers lease 256 lanes; the lease columns are dropped and `processing` deliveries become `pending`. | Stop 0.1.0 workers first; `setup()` converts the data. | | Deduplication relies on completed rows. | Every event remembers the handlers it ever had a delivery for, so a redelivered event runs no handler twice. | None. Keep `retention.completed` longer than any broker redelivery window. | | No notifications. | PostgreSQL `LISTEN`/`NOTIFY` wakes other processes. | None, or `wakeUp: 'poll'` behind a pooler. | | Statements are parsed on every call. | Repeated statements are prepared per connection. | None, or `preparedStatements: false` behind PgBouncer before 1.21. | ### Before you start 1. **Back up the database.** There is no down migration. 2. **Rehearse on a copy** of production data and time it. The PostgreSQL upgrade rewrites every `deliveries` row once. 3. **Purge old completed deliveries** to shorten that rewrite: set a short `retention.completed` and run `bus.purgeExpired()` while still on 0.1.0. 4. **Update every package to the same version**, for example `pnpm add @relaybus/core@0.2.0 @relaybus/nestjs@0.2.0 @relaybus/postgres@0.2.0` (and `mongodb`, `kafka`, `rabbitmq`, `testing` if you use them). ### Step 1: stop every 0.1.0 worker Scale the processes that run a dispatcher (API processes with the dispatcher enabled, worker fleets) to zero. Running 0.1.0 and 0.2.0 together is not supported: 0.1.0 code fails on the dropped columns. ### Step 2: upgrade the schema Choose one. #### Option A: let `setup()` do it (default) Start one 0.2.0 process with `setupStore: true` (the default). `setup()` runs in one transaction under an advisory lock and is safe to run again. In order, on PostgreSQL it: 1. Detects a 0.1.0 table by its `deliveries.lease_token` column. 2. Adds `events.handlers`, filled from the existing deliveries (completed ones included, so nothing that completed under 0.1.0 runs again). 3. Adds `deliveries.lane` with the same hash new deliveries use, turns `processing` deliveries into `pending` (the attempt stays counted, as a lost lease would count it) and drops `lease_owner`, `lease_token` and `lease_expires_at`. 4. Drops the indexes that no longer exist, creates the new ones and seeds the 256 `lanes` rows. Deliveries that were `completed` stay as `completed` rows and are purged by `retention.completed`, or kept forever with `retention: { completed: false }`. New completions are deleted. #### Option B: run the SQL with your own migration tool When your application role has no DDL rights, or you apply migrations from CI, render the script and run it as **one transaction with the workers stopped**: ```ts import { createPostgresSchemaSql } from '@relaybus/postgres'; // schema and tablePrefix must match your store options console.log(createPostgresSchemaSql({ schema: 'app', tablePrefix: 'bus_' })); ``` Then start 0.2.0 with `setupStore: false`, so the processes only need DML rights. #### MongoDB `setup()` upgrades the database in place and repeatably: `processing` deliveries become `pending` and due at once (the attempt stays counted), per-delivery lease fields are removed, every delivery gets its `laneId` and every event its `handlers` and `createdAt` (in batches of 1,000), the old indexes are dropped and the 256 lane documents are created. The lane documents mark the upgrade as done, so later runs skip it. `completed` documents left by 0.1.0 stay until `purge({ completedBefore })` removes them. There is no SQL to run: if your application role cannot create collections, call `setup()` once from a deployment script with a role that can. ### Step 3: keep completion history (optional) If an audit, a dashboard or a test reads completed deliveries, opt in: ```ts new PostgresEventStore({ pool, completionHistory: true }); new MongoEventStore({ client, database: 'orders', completionHistory: true }); new InMemoryEventStore({ retainCompleted: true }); // tests ``` With it, each completion also inserts one append-only row into `completed_deliveries` (a collection on MongoDB) in the same transaction that deletes the delivery. `listDeliveries({ status: 'completed' })` and `statistics().completed` read it, and `retention.completed` purges it. The cost is one more heap and index write per delivery, so leave it off on hot paths. Use the same value in every process. Completed rows that 0.1.0 left in `deliveries` keep being listed until retention purges them, whatever this option says. ### Step 4: start 0.2.0 and verify Start the workers, one at a time if you want to watch, then check: ```ts console.log(await bus.statistics()); // pending falls, dead is unchanged, nothing stays in processing ``` ```sql SELECT count(*) FROM relaybus_lanes; -- 256 SELECT count(*) FROM relaybus_deliveries WHERE lane IS NULL; -- 0 ``` Dead letters from 0.1.0 are still listed and can be requeued as before. ### PgBouncer and other poolers A pooler in transaction mode gives each transaction a different server session, and 0.2.0 uses two things that need a session: `LISTEN` and prepared statements. | Setup | Settings | | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | | Direct to PostgreSQL, or PgBouncer in session mode | Defaults. | | PgBouncer 1.21 or newer in transaction mode with `max_prepared_statements > 0` | `wakeUp: 'poll'` (no `LISTEN` through a transaction pooler); prepared statements work. | | PgBouncer **before 1.21** in transaction mode, or `max_prepared_statements = 0` | `preparedStatements: false` and `wakeUp: 'poll'`. | ```ts new PostgresEventStore({ pool, // connects through PgBouncer wakeUp: 'poll', preparedStatements: false, }); ``` If you forget `preparedStatements: false`, the first statement that lands on another session fails once, the store stops preparing and logs `[PgExecutor](query) The server does not keep prepared statements per connection`. When you poll, shorten `dispatcher.pollInterval` to the latency you need. See [Tuning](https://berkcansavur.github.io/relaybus/tuning#pgbouncer-in-transaction-mode). ### New optional APIs None of these is required to upgrade. - **Event catalogs:** `defineCatalog`, `HandlersOf`, `defineHandlers`, `bus.forCatalog`, `bus.subscribeCatalog`, `consume(...)` and `produce(...)` roles, and `@EventConsumer` in `@relaybus/nestjs`. [Guide](https://berkcansavur.github.io/relaybus/concepts/event-catalogs). - **Foreign messages:** `decode: plainJsonDecoder(...)` and `decode: decodeCloudEvent` on the Kafka and RabbitMQ transports. [Guide](https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services). - **Store options:** `wakeUp`, `preparedStatements`, `completionHistory`. [Tuning](https://berkcansavur.github.io/relaybus/tuning). ### Rollback Going back to 0.1.0 needs a restore of the backup you took before step 2. Keep it until you are happy with 0.2.0 in production. ### Related - [Releases: what is new and known issues](https://berkcansavur.github.io/relaybus/releases) - [Performance: 0.2.0 vs 0.1.0](https://berkcansavur.github.io/relaybus/performance) - [@relaybus/postgres: Upgrading from 0.1.0](https://berkcansavur.github.io/relaybus/packages/postgres#upgrading-from-0-1-0) - [@relaybus/mongodb: Upgrading from 0.1.0](https://berkcansavur.github.io/relaybus/packages/mongodb#upgrading-from-0-1-0) --- Source: https://berkcansavur.github.io/relaybus/stability # Stability This page says what you can rely on when you upgrade relaybus, and what you cannot. [Compatibility](https://berkcansavur.github.io/relaybus/compatibility) lists what is tested; [Releases](https://berkcansavur.github.io/relaybus/releases) lists what changed. ## Versioning All `@relaybus/*` packages share one version and are released together. Install them at the same version. | Version | A breaking change bumps | A new feature bumps | A fix bumps | | --------------- | -------------------------- | ------------------- | ----------- | | `0.x` | the minor (`0.2` to `0.3`) | the minor | the patch | | `1.x` and later | the major | the minor | the patch | Below 1.0.0 a minor release may contain breaking changes. Each one is marked `BREAKING CHANGE` in the commit, listed under "Breaking changes" in [Releases](https://berkcansavur.github.io/relaybus/releases), and has a step in [Upgrading](https://berkcansavur.github.io/relaybus/upgrading). From 1.0.0 on, only a major release breaks the public API, the wire format or a documented behaviour. ## What is public API The public API is what the API snapshot test pins: the exported names of the entry point of every package (`@relaybus/core`, `nestjs`, `postgres`, `mongodb`, `kafka`, `rabbitmq`, `testing`, `eslint-plugin`, `otel`), each as a value, a type or both. The snapshot is committed, so a change to the exported surface shows up in review and is always deliberate. Alongside the names, the documented behaviour of those exports is public: option defaults, error classes and their `code`, the guarantees on [Guarantees](https://berkcansavur.github.io/relaybus/guarantees). Not covered by the policy: - **`@relaybus/core/internal`.** A subpath shared by the packages (validators, decoder helpers, timers). It is pinned by a snapshot too, but only so the packages stay consistent; it is undocumented and can change in any release. Do not import it. - **Anything not exported**, deep imports into `dist/`, and the contents of `src/`. - **Log message bodies.** Lines start with `[ClassName](methodName)`, but wording can change. Branch on error `code`, not on log text or error messages. - **Error messages.** Codes are stable; messages are not. - **Tables, collections, indexes and statements.** You may read the tables for debugging; write only through relaybus. - **Benchmark numbers** and the `benchmarks/` harness. ## Deprecation policy An API is deprecated, with a warning in the types, for at least one minor release before it is removed, and only a major release removes it. Below 1.0.0 the removal waits for 1.0.0. - A deprecated API keeps working and keeps its behaviour. It is marked `@deprecated` in the types, so editors strike it through, and its documentation names the replacement. - Deprecations are listed in [Releases](https://berkcansavur.github.io/relaybus/releases) with the version that removes them. Today: `@Subscribe`, `@EventConsumer` and the inbound-only `decode` option of the Kafka and RabbitMQ transports, all removed in 1.0.0. - Error `code` values are never renamed or reused once released. ## Wire format What relaybus writes to a broker is a compatibility surface of its own, because other services read it. - **CloudEvents binary mode is the default** for Kafka and RabbitMQ, and the default codec writes byte for byte the same message as the release before it. Golden tests pin the bytes, so a refactoring cannot change them silently. - Changing the default wire format is a breaking change: 1.x does it only in a major release, and 0.x says so as a breaking change. - New wire formats are opt-in through `codec` (`cloudEventsStructured`, `jsonEnvelope`, or your own). - New headers may be added (for example the error name and code on dead letters, or `traceparent` when you enable a tracer). Consumers must ignore headers they do not know. ## Database schema - A new release may change the store schema. It does so in place: `setup()` upgrades an older schema inside one transaction under an advisory lock, and is safe to run again. - **Applying the schema yourself.** If the application role has no DDL rights, render the script with `createPostgresSchemaSql({ schema, tablePrefix })` and run it with your migration tool, as one transaction with the workers stopped. The release notes say when the script changed. - There is no down migration. Back up before upgrading and rehearse on a copy. - 0.3.0 changed no table, column or index. - Running two versions against one database is supported only where [Upgrading](https://berkcansavur.github.io/relaybus/upgrading) says so (0.2 and 0.3 during a rolling deploy). ## Runtimes | Runtime | Policy | | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Node.js | Node.js 22.12 and newer (`engines`); CI runs the tests on Node 22 and 24. Dropping a Node version is announced in [Releases](https://berkcansavur.github.io/relaybus/releases) and, from 1.0.0, happens only in a major release. | | NestJS | The majors in the peer dependency range (`^11 | | ^12`), each tested in CI. | | PostgreSQL, MongoDB | The versions in [Compatibility](https://berkcansavur.github.io/relaybus/compatibility). Dropping a server version is announced in [Releases](https://berkcansavur.github.io/relaybus/releases). | | Brokers and clients | As listed in [Compatibility](https://berkcansavur.github.io/relaybus/compatibility). | Bun and Deno are not tested and not supported. ## Reporting a break If a release broke something this page says it should not have, that is a bug: open an issue on [GitHub](https://github.com/berkcansavur/relaybus/issues). Security problems follow the [security policy](https://github.com/berkcansavur/relaybus/blob/main/SECURITY.md). --- Source: https://berkcansavur.github.io/relaybus/compatibility # Compatibility What is tested, what is supported but not tested, and what is not supported. "Tested" means a CI job or an integration test in the repository runs against it; the [Stability](https://berkcansavur.github.io/relaybus/stability) policy says how this list changes. ## Runtimes | | Status | Evidence | | ---------------------------- | ------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | Node.js 22 (22.12 and newer) | tested | CI test matrix `22.x` and `24.x` | | Node.js 24 | tested | CI test matrix | | CommonJS and ESM | built and type-checked for both | every package ships `import` and `require` entry points with their own types; `publint` and `attw` check them in CI | | NestJS 11 and 12 | tested | CI job `nestjs compat` re-resolves `@nestjs/*` to each major and runs the typecheck and the `@relaybus/nestjs` tests | | Bun, Deno | untested, not supported | | ## Stores | Store | Versions | Notes | | ------------------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | PostgreSQL | 13 and newer; integration tests run on 13 and 17 | `pg` is the primary client and the only one with prepared statements. | | `pg` pool or client | tested | `PostgresEventStore({ pool })` | | Kysely | integration tested | `createKyselyExecutor` | | Knex | integration tested | | | TypeORM | integration tested | | | Drizzle | integration tested | | | Prisma | integration tested (Client 5.22, 6.19 and 7.10, `@prisma/adapter-pg`) | Full contract suite on a generated client. Give the store your `pg` pool as well for prepared statements and wake-ups at commit. | | MongoDB | 6.0 and newer, integration tests on 7 | **Replica set or sharded cluster only**: relaybus uses multi-document transactions. A single-node replica set is enough for development. Driver `mongodb` 6 or 7; Mongoose sessions work. | ## Transports | Transport | Tested | Notes | | ------------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Kafka with `kafkajs` 2.2.x | integration tested against a real broker (`confluentinc/cp-kafka:7.5.0`), 2.2.4 | `kafkajs` is **unmaintained**. It stays supported and is the default `client`; prefer Confluent for new services. | | Kafka with `@confluentinc/kafka-javascript` | integration tested against the same broker, 1.11.0 (KafkaJS-compatible API) | **Recommended**: maintained and librdkafka based. Pass `client: 'confluent'`; the transport puts its settings in the `kafkaJS` blocks and follows the group through `rebalance_cb`. Peer range `^1.11.0`. See [Choosing a client](https://berkcansavur.github.io/relaybus/packages/kafka#choosing-a-client). | | RabbitMQ | integration tested on RabbitMQ 4 (`rabbitmq:4-alpine`); 3.13 or newer is required | **Quorum queues** are used for the queue and the dead-letter queue. `amqplib` 0.10 or newer. | ### Codecs | Codec | Kafka | RabbitMQ | | ----------------------------- | -------------------- | -------------------- | | `cloudEventsBinary` (default) | tested, golden bytes | tested, golden bytes | | `cloudEventsStructured` | tested | tested | | `jsonEnvelope({ ... })` | tested | tested | | `{ encode, decode }` (custom) | tested | tested | ## Explicitly unsupported These are not planned for the current release line; no store or transport exists for them. | | Why / what to use | | ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | MySQL, SQL Server, SQLite | No store. relaybus needs `SELECT ... FOR UPDATE SKIP LOCKED`-style claims, advisory locks and a transactional outbox in your database; PostgreSQL and MongoDB are the stores. | | MongoDB standalone | No multi-document transactions. Use a replica set. | | Amazon SQS and SNS, NATS, Google Pub/Sub, Redis Streams | No transport. Implement the small `Transport` interface (`publish`, `subscribe`, `close`) yourself if you need one; it is not tested by relaybus. | | Bun, Deno | Untested. | ## Serverless and short-lived processes relaybus delivers from a long-running dispatcher: it claims deliveries on a poll and, on PostgreSQL, wakes on `LISTEN`/`NOTIFY`, and it holds leases while a handler runs. A process that is frozen or killed between requests cannot do that. - **Publishing from a function works**: publish inside your database transaction and let the commit make the event durable. Set `dispatcher: { enabled: false }` in the function and run the dispatcher in a long-running worker ([Tuning](https://berkcansavur.github.io/relaybus/tuning)). - **Dispatching from a function** is possible only by calling `bus.runOnce()` per invocation (one batch) and finishing before the platform freezes the process. Expect higher latency, no `NOTIFY` wake-ups, and leases that expire when an invocation is cut short, which makes the delivery run again (at-least-once holds). - Behind a connection pooler in transaction mode, use the settings in [Tuning](https://berkcansavur.github.io/relaybus/tuning) (`wakeUp: 'poll'`, `preparedStatements: false` where needed). ## Where this is checked CI (`.github/workflows/ci.yml`): `quality` (Prettier, ESLint, typecheck, performance budgets), `test` on Node 22 and 24 with 100 % coverage thresholds, `build` (all packages, `publint`, `attw`), `nestjs-compat` on NestJS 11 and 12. Integration tests start their databases and brokers with Testcontainers. --- Source: https://berkcansavur.github.io/relaybus/performance # Performance: 0.2.0 vs 0.1.0 On PostgreSQL, 0.2.0 publishes 1.1 to 1.9 times faster than 0.1.0 and dispatches 1.3 to 18 times faster (one cell is slower, see below). On MongoDB it dispatches 2 to 26 times faster and publishes at the same speed. It writes about half the WAL per delivery on PostgreSQL. Every sample was checked for lost, duplicate and out-of-order deliveries: there were none. Read the numbers as ratios on one laptop, not as capacity planning. Where a cell has a single sample the table says so. - [How it was measured](#how-it-was-measured) - [Publish](#publish) - [Dispatch on PostgreSQL](#dispatch-on-postgresql) - [Dispatch on MongoDB](#dispatch-on-mongodb) - [Mixed load and latency](#mixed-load-and-latency) - [What the database does per delivery](#what-the-database-does-per-delivery) - [Where it is not better](#where-it-is-not-better) - [0.5.0 vs 0.4.1](#_0-5-0-vs-0-4-1) - [0.4.0 vs 0.3.1](#_0-4-0-vs-0-3-1) - [0.3.0 vs 0.2.0](#_0-3-0-vs-0-2-0) - [Deep dives](#deep-dives) - [Reproduce it](#reproduce-it) ## How it was measured | Aspect | What was done | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Same machine | Both versions ran on one machine in one run (fingerprint `6ba3daf9b9a1`), against the same database images. | | Interleaved | For every scenario the base (0.1.0) and the candidate (0.2.0) alternate: base first, then candidate, then candidate first, so drift of the machine (heat, other processes) hits both alike. | | Fixed windows | Each sample is a warm-up plus a fixed steady-state window (publish 3 s + 15 s, dispatch from a preloaded database, mixed 4 s + 20 s). Nothing waits for a drain, so a slow side cannot hide behind a long run. | | Real processes | Every publisher and every dispatcher instance is its own Node process with its own pool and bus. Handlers are no-ops, so the numbers are the cost of the bus and the store, not of your handlers. | | Database counters | Before each sample the database statistics are reset; afterwards statements, WAL bytes and records, rows inserted, updated and deleted (PostgreSQL) or operations and oplog bytes (MongoDB) are divided by the events or deliveries actually counted in the database. These move when the code changes, not when the laptop is busy. | | Exactness | After every dispatch and mixed sample the database is checked: no delivery lost, none handled twice without a lease loss, per-key order held, nothing dead. Publish samples must hold exactly `events x handlers` deliveries. **All samples in this run were valid: 0 lost, 0 duplicate, 0 out of order.** | | Noise control | A fixed CPU spin is timed before each sample; a sample that ran more than 1.25 times slower than the quietest one is discarded together with its pair when two clean pairs remain. | Dispatcher configurations: `defaults` are the `RelayBus` defaults (concurrency 10, batch size 10, poll 1 s, `pg` pool 10, MongoDB pool 100). `mid` is concurrency 50, batch 200, pool 20. `tuned` is concurrency 200, batch 200, pool 40, poll 50 ms. `Ni` means N dispatcher processes. ### The machine | Part | Value | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | Host | Apple M4, 10 cores (4 performance), 16 GiB, macOS (Darwin 24.6.0) | | Node.js | v24.3.0 | | Docker | Docker Desktop, 10 CPUs and 8 GiB for the VM, server 28.5.1 | | PostgreSQL | 17.11 (`postgres:17-alpine`): `fsync=on`, `synchronous_commit=on`, `full_page_writes=on`, `shared_buffers=1GB`, `max_wal_size=4GB`, `wal_level=replica` | | MongoDB | 7.0.45 (`mongo:7`), single-node replica set | | Round trip | PostgreSQL p50 0.20 ms, p99 0.35 ms; MongoDB p50 0.25 ms, p99 0.66 ms | Docker Desktop runs the databases in a Linux VM that shares the physical cores with the Node workers, and fsync goes to a virtual disk. Absolute numbers are a lower bound for a dedicated database server; the ratios are the useful part. The base is `main` at `d9ca3be` (0.1.0), the candidate is `eb3fbd7` (the 0.2.0 branch); the run took 38.9 minutes. ### How much to trust a number A cell with three samples per side has a 95 % confidence interval and a verdict. A cell with **one sample per side** has neither: its ratio is **indicative**, and a difference below roughly 10 % cannot be told from noise. Database counters (WAL, rows written) are exact either way, which is why they, not throughput, gate a change. In the tables, a **3** in the _samples_ column means a confirmed result and a **1** means indicative. ## Publish Events per second written by publishing processes, one handler unless noted. `P` is publishers, `batch` is events per `publishAll` call. | Scenario | 0.1.0 events/s | 0.2.0 events/s | Ratio | Samples | | --------------------------------------------- | -------------: | -------------: | ----: | :-----: | | PostgreSQL, P=1, batch 1 (*) | ~1,310 | ~1,650 | 1.26x | 1 | | PostgreSQL, P=1, batch 100 | 16,740 | 20,153 | 1.20x | 1 | | PostgreSQL, P=8, batch 1 | 3,703 | 4,603 | 1.24x | 1 | | PostgreSQL, P=8, batch 100 | 63,853 | 73,533 | 1.15x | 3 | | PostgreSQL, P=32, batch 1 (*) | ~5,150 | ~6,200 | 1.20x | 1 | | PostgreSQL, P=32, batch 100 | 65,520 | 76,160 | 1.16x | 1 | | PostgreSQL, 4 processes x 8, batch 100 | 59,213 | 66,967 | 1.13x | 1 | | PostgreSQL, P=8, batch 1, 5 handlers | 2,408 | 2,893 | 1.20x | 1 | | PostgreSQL, P=8, batch 100, 5 handlers | 20,987 | 29,060 | 1.39x | 3 | | PostgreSQL, P=8, batch 1, 1,000 distinct keys | 2,482 | 3,982 | 1.60x | 1 | | PostgreSQL, P=8, batch 1, 10 hot keys | 2,615 | 3,975 | 1.52x | 1 | | PostgreSQL, P=8, batch 1, 1 hot key | 927 | 1,787 | 1.93x | 1 | | MongoDB, P=8, batch 1 | 1,553 | 1,599 | 1.03x | 1 | | MongoDB, P=8, batch 100 | 31,600 | 31,167 | 0.98x | 3 | | MongoDB, P=32, batch 100 | 24,460 | 31,387 | 1.28x | 1 | | MongoDB, P=8, batch 100, 5 handlers | 10,260 | 13,633 | 1.33x | 1 | | MongoDB, P=8, batch 1, 10 hot keys | 668 | 912 | 1.37x | 1 | (*) These three cells were slower than 0.1.0 on the first run of the 0.2.0 branch. The values shown were re-measured after the prepared-statement fix described in [the publish path](https://berkcansavur.github.io/relaybus/performance/publish-path), with fewer samples than the other cells. The main 38.9-minute run, which already contained the fix, gave 1,281 to 1,696, 5,052 to 6,440 and 2,056 to 3,716 for the three cells (P=1, P=32 and the dispatch cell below). In words: single-event publishes on PostgreSQL are about a quarter faster, large batches 15 to 20 % faster, and fan-out to five handlers up to 39 % faster, because 0.2.0 writes fewer indexes and one statement per publish. Publishing to a hot ordering key is nearly twice as fast, since the key's sequence counter is now updated in one statement. MongoDB publish is unchanged within noise. Publish latency (p99 of one publishing transaction, PostgreSQL, P=8, batch 100): 27.3 ms to 19.8 ms (confirmed, three samples). ## Dispatch on PostgreSQL Deliveries completed per second, from a preloaded backlog of unkeyed deliveries unless noted. Handlers do nothing. | Scenario | 0.1.0 deliveries/s | 0.2.0 deliveries/s | Ratio | Samples | | ------------------------------------------- | -----------------: | -----------------: | ---------: | :-----: | | 1 process, `defaults` (*) | ~3,100 | ~4,000 | 1.29x | 1 | | 1 process, `mid` | 3,136 | 10,273 | 3.28x | 1 | | 1 process, `tuned` | 9,329 | 14,348 | 1.82x | 3 | | 4 processes, `defaults` | 2,334 | 10,043 | 4.30x | 1 | | 4 processes, `mid` | 2,805 | 32,306 | 11.52x | 1 | | 4 processes, `tuned` | 3,242 | 55,629 | 17.16x | 1 | | 8 processes, `defaults` | 5,478 | 17,371 | 3.17x | 1 | | 8 processes, `mid` | 8,750 | 45,632 | 5.22x | 1 | | 8 processes, `tuned` | 9,736 | 62,937 | 6.46x | 1 | | Fan-out to 5 handlers, 1 process, `tuned` | 8,220 | 17,838 | 2.17x | 1 | | Fan-out to 5 handlers, 4 processes, `tuned` | 2,964 | 54,474 | 18.38x | 1 | | 1,000 distinct keys, 1 process, `tuned` | 2,307 | 7,332 | 3.18x | 1 | | 1,000 distinct keys, 4 processes, `tuned` | ~2,690–3,300 | ~5,200–5,300 | 1.61–1.93x | 2 | | 10 hot keys, 1 process, `tuned` | 1,110 | 1,565 | 1.41x | 1 | (*) Re-measured after the prepared-statement fix, with fewer samples than the other cells; the main run gave 2,056 to 3,716. Why 0.1.0 does not scale with processes (2,334 at four, 3,242 with 200 handlers): every delivery was updated three times, so processes spent their time rewriting the same rows and waiting for each other. 0.2.0 workers lease lanes instead of rows, so adding a process adds throughput until the database is the limit. The single-sample cells above 10x are real in direction, but their exact multiples are noisy; the three-sample cell (1 process, `tuned`, 1.82x, interval 1.38 to 4.29) is the confirmed one. ## Dispatch on MongoDB | Scenario | 0.1.0 deliveries/s | 0.2.0 deliveries/s | Ratio | Samples | | ----------------------------------------- | -----------------: | -----------------: | -----: | :-----: | | 1 process, `defaults` | 485 | 997 | 2.06x | 1 | | 1 process, `tuned` | 560 | 6,281 | 11.12x | 3 | | 4 processes, `defaults` | 505 | 3,451 | 6.84x | 1 | | 4 processes, `tuned` | 587 | 15,437 | 26.31x | 1 | | Fan-out to 5 handlers, 1 process, `tuned` | 587 | 8,599 | 14.66x | 1 | | 1,000 distinct keys, 1 process, `tuned` | 173 | 1,955 | 11.28x | 1 | | 10 hot keys, 1 process, `tuned` | 177 | 347 | 1.96x | 1 | 0.1.0 issued one conditional update per claim and one per completion, about three oplog entries per delivery. 0.2.0 claims a whole batch with one write to the lane documents and completes it with one bulk delete. ## Mixed load and latency Publishing at a fixed rate while dispatching, measuring commit-to-handler latency (end to end). One sample per side, so indicative. | Scenario | Accepted events/s 0.1.0 to 0.2.0 | p99 latency 0.1.0 | p99 latency 0.2.0 | | ------------------------------- | -------------------------------: | ----------------: | ----------------: | | PostgreSQL, 1,000/s, `defaults` | 1,000 / 1,000 | 991 ms | 10.5 ms | | PostgreSQL, 1,000/s, `tuned` | 1,001 / 1,000 | 59.9 ms | 9.8 ms | | PostgreSQL, 5,000/s, `tuned` | 2,522 / 4,099 | 114 ms | 524 ms | | MongoDB, 1,000/s, `tuned` | 1,000 / 1,000 | 18.1 s | 389 ms | | MongoDB, 5,000/s, `tuned` | 1,756 / 1,741 | 21.0 s | 8.0 s | At 1,000 events per second both versions keep up and 0.2.0 delivers in about a hundredth of the time on `defaults`. At 5,000 per second on a laptop both versions are past saturation and shed load; on PostgreSQL 0.2.0 accepts 1.6 times more events but its p99 is higher than 0.1.0's at that higher load. See [known issues](https://berkcansavur.github.io/relaybus/releases#known-issues). ## What the database does per delivery These counters are exact, not statistical. PostgreSQL dispatch, per delivery: | Counter | 0.1.0 | 0.2.0 | | --------------------------------- | -------------- | ----------------------------------- | | `deliveries` rows updated | 2 | 0 | | `deliveries` rows deleted | 0 | 1 | | `lanes` rows updated | n/a (no lanes) | about 0.05 (one lane row per batch) | | `event_completions` rows inserted | n/a | 1 | | WAL bytes (1 process, `tuned`) | 2,336 | 1,325 (0.57x) | | WAL bytes (4 processes, `tuned`) | 2,501 | 1,316 (0.53x) | | WAL bytes (fan-out 5, 1 process) | 2,300 | 576 (0.25x) | | WAL records (1 process, `tuned`) | - | 0.38x of 0.1.0 | | WAL bytes (1 process, `defaults`) | 2,240 | 3,129 (1.40x, see below) | Publish, per event: WAL 1,758 to 1,490 bytes (0.85x), 14.2 to 11.2 WAL records, statements 5 to 4.1 for a single-event publish. MongoDB dispatch writes 0.17 to 0.72 times the oplog entries and 0.22 to 0.61 times the oplog bytes. **Why the benchmark gate failed, and why that is expected.** The gate flags `deliveries.delPerUnit` as "regressed from zero" in every dispatch cell, and `deletePerUnit` on MongoDB: 0.1.0 never deleted a delivery, 0.2.0 deletes each one on completion instead of updating it twice. That is the intended change, not a regression; the counter that matters is the total WAL and write volume, which fell. The one real counter regression is WAL per delivery with `defaults` on one process (1.40x): the completion inserts an `event_completions` row whose foreign-key check locks the event row, and the first change to a page after a checkpoint writes the whole page to WAL. It fades as pages are touched again within a checkpoint cycle and did not reduce throughput; see [the publish path](https://berkcansavur.github.io/relaybus/performance/publish-path#dispatch-one-process-with-default-settings-pg-disp-1i-defaults). ### Write amplification and dead tuples In 0.1.0 a delivery row was updated three times (claim, lease renewals, completion). Each update of an indexed column such as `status` writes a new row version plus an entry in every index, and leaves a dead tuple for autovacuum. Measured on 0.1.0: 0 % HOT updates and a steady stream of dead tuples and dead index entries. In 0.2.0 the same delivery is inserted once and deleted once; claiming and renewing update only one of 256 lane rows per batch, and those updates are 100 % HOT. A delete leaves one dead tuple per delivery instead of three and no dead index entries from updates. The tables are created with `autovacuum_vacuum_scale_factor = 0.02`; alert on `n_dead_tup` for sustained thousands of deliveries per second. The full explanation, with `pageinspect` evidence and diagrams, is in [Write amplification in the delivery path](https://berkcansavur.github.io/relaybus/performance/write-amplification). ## Where it is not better - **Keyed dispatch on four processes with 1,000 distinct keys:** the first full run showed 0.66x, but that came from one unusually fast 0.1.0 sample (7,072/s against 2,690–3,300/s in the re-runs). Two re-runs with 0.1.0 measured alongside put 0.2.0 at 1.61x and 1.93x, so the table shows the re-run range; it is not a regression. - **MongoDB batch-100 publish:** 31,600 to 31,167 events/s (0.98x, three samples): no change, and a higher p99 for the transaction (79 ms to 126 ms, interval inconclusive). - **Mixed load near saturation:** p99 can be higher than 0.1.0's, see above. - **Single sample cells** may move by 10 % or more in either direction on a re-run. The directions are consistent with the exact counters, which is why they are listed. ## 0.5.0 vs 0.4.1 0.5.0 adds catalog routes, wire names and transactions at an isolation level; without `isolationLevel` a transaction runs the statements of 0.4.1. The numbers agree: publish and dispatch on PostgreSQL are within noise of 0.4.1, and the statements per event and per delivery are identical. Every sample was valid. Machine fingerprint `2c551bf14f01` (the setup has changed since the runs above), candidate `adadab2` against `v0.4.1`, interleaved. Publish was measured with **6 samples per side**, dispatch with **3**. The machine was busy (load average 5 to 28 during the publish run), which is why single pairs move by up to 30 % either way. Throughput is candidate / base, the median of the per-pair ratios: | Scenario | Ratio | Interval (95 %) | Verdict | | -------------------------------------------- | ----: | --------------- | ------------ | | PostgreSQL publish, P=8, 1 event per call | 0.998 | 0.85 to 1.05 | inconclusive | | PostgreSQL publish, P=8, 100 per call | 1.006 | 0.84 to 1.20 | inconclusive | | PostgreSQL publish, 100 per call, 5 handlers | 0.921 | 0.84 to 1.18 | inconclusive | | PostgreSQL dispatch, 1 instance, `tuned` | 1.019 | - | no change | | PostgreSQL dispatch, 4 instances, `tuned` | 1.014 | - | within noise | | PostgreSQL dispatch, 10 hot keys, `tuned` | 0.964 | - | within noise | Database counters: publish counters (18 per cell) are all within 3 %, statements per event 1.000x. On the dispatch cells statements per delivery are 0.997x to 1.001x, WAL bytes per delivery 0.98x to 1.02x, and `lanes` updates per delivery 1.09x (1 instance), 1.08x (4 instances) and 1.00x (hot keys), inside the 15 % that this counter is gated at (it follows the timing of concurrent claims). The dispatch intervals are left out: that run stopped at the MongoDB cells, so its report was not written and the ratios above are computed from its stored samples. MongoDB was not measured. On this machine the harness could not load the MongoDB template of the 0.4.1 side: a 500-event transaction of the loader ran past the 5 s transaction budget, three times. The candidate side was never reached, so this says nothing about 0.5.0 on MongoDB; the MongoDB store has no change in this release. ### Changes to the gate From this release the quick gate generates the same event ids and keys on both sides (a seeded generator per publisher), so the lane layout no longer differs between base and candidate; the dispatch cells measure a fixed amount of work instead of a fixed time; counters of interleaved runs are judged per pair; and `lanes` updates per delivery gate at 15 % instead of 3 %. Identical code now passes against itself (two runs), and a deliberate regression (one extra `lanes` update per completion batch) fails it at +97 % (1 instance) and +85 % (4 instances). ### Reports The raw samples and environment are in `benchmarks/results/runs/`: - `2026-10-27T10-16-43-144Z`: the publish cells, 6 pairs (report written, gate PASS). - `2026-10-27T10-05-19-668Z`: the PostgreSQL dispatch cells, 3 pairs (stopped at the MongoDB cells, no report). ## 0.4.0 vs 0.3.1 0.4.0 adds features (a `pool` for ORM stores, wake-up at the caller's commit, new Kafka options) and leaves the delivery path alone, and the numbers agree: publish and dispatch are within noise of 0.3.1 on PostgreSQL and MongoDB, and the statements per delivery are identical. Every sample was valid (6 of 6 per cell). Same machine as above (fingerprint `f2d136dc9ad0`), candidate `a3b56bc` against `v0.3.1`, interleaved, **3 samples per side**. Throughput is candidate / base: | Scenario | Ratio | Interval (95 %) | Verdict | | -------------------------------------------- | ----: | --------------- | ------------ | | PostgreSQL publish, P=8, 1 event per call | 1.011 | 0.96 to 1.05 | no change | | PostgreSQL publish, P=8, 100 per call | 1.005 | 1.00 to 1.37 | inconclusive | | PostgreSQL publish, 100 per call, 5 handlers | 1.033 | 0.55 to 1.88 | inconclusive | | MongoDB publish, P=8, 100 per call | 0.984 | 0.97 to 1.17 | within noise | | PostgreSQL dispatch, 1 instance, `tuned` | 1.062 | 1.00 to 1.47 | inconclusive | | PostgreSQL dispatch, 4 instances, `tuned` | 1.048 | 0.91 to 1.13 | inconclusive | | PostgreSQL dispatch, 10 hot keys, `tuned` | 0.996 | 0.99 to 1.00 | no change | | MongoDB dispatch, 1 instance, `tuned` | 1.009 | 0.97 to 1.20 | inconclusive | Database counters per delivery: one delivery delete and one completion insert, as in 0.3.1. The PostgreSQL dispatch cells show `lanes` updates per delivery at 0.97x (1 instance) and 0.92x (4 instances) and WAL bytes at 0.95x and 0.88x; hot keys show WAL bytes at 1.04x and WAL records at 1.03x. Publish counters (18 per PostgreSQL cell, 6 per MongoDB cell) are all within 3 %. ### About the two counter flags The benchmark gate flagged the 10 hot keys cell for WAL bytes (4.0 %) and WAL records (3.1 %) per delivery, so the run is reported as a gate failure. We do not read that as a regression, for this reason: the gate compares counters of two databases that the harness generates randomly, one preloaded for each side, and each side dispatches its own. To see how large that difference is on identical code, the harness ran `v0.3.1` against itself (A/A, `2026-10-25T18-59-07-978Z`). That run also failed the gate, with `lanes.updPerUnit` 9.5 % higher (1.09x) on a side that has no change at all. A separate candidate run of the same two dispatch cells (`2026-10-25T18-55-40-858Z`) passed: `lanes.updPerUnit` 0.90x and 18 counters within 3 % on the hot keys cell. So in the one A/A run the counter gate moved by 9.5 % on identical code, well above its 3 % threshold; that is one run, not a measured noise floor. What this does and does not show: a 3 to 4 % difference in one cell is inside that floor and cannot be told from noise with 3 samples, and the statements per delivery, which do not depend on the preloaded data, are identical. It does not prove that WAL per delivery is exactly the same, only that nothing here separates it from 0.3.1. ### Reports The raw reports, samples and environment of the three runs are in `benchmarks/results/runs/`: - `2026-10-25T19-02-48-248Z`: the interleaved run of all the cells above (the quick gate set). - `2026-10-25T18-59-07-978Z`: the A/A run, `v0.3.1` against `v0.3.1`, two dispatch cells. - `2026-10-25T18-55-40-858Z`: the candidate against `v0.3.1` on the same two dispatch cells. ## 0.3.0 vs 0.2.0 0.3.0 is a code-quality release, and the numbers say it cost nothing: publish and MongoDB are unchanged within noise, most PostgreSQL dispatch cells are within about 10 to 20 % either way (which one sample cannot resolve), and keyed dispatch on PostgreSQL is 1.78 times faster in two cells. Every sample was checked for lost, duplicate and out-of-order deliveries: 0 of each. Same machine, same method and same database images as above (fingerprint `0c08a14e7a55`). Two runs feed the tables: - **Full run**, all cells, 39 minutes, candidate `a8587ff` against `v0.2.0`, interleaved. Each cell has **1 sample per side**, except where the table says 3. Every sample was valid except the two of PostgreSQL fan-out to 5 handlers on 4 processes (the preloaded backlog drained 0.5 s into the window on both sides), which are left out. - **Targeted run** for the publish and mixed-load cells, candidate `5575bc4`, **2 samples per side**. The full run was taken before two publish fixes (`a13fb7e`, `2fb3cde`: the transaction budget no longer costs a statement per publish), so these cells were measured again. The 0.2.0 side of this run reused the stored 0.2.0 samples; it was not measured alongside the candidate, so machine drift is not cancelled out. ### Publish on PostgreSQL (targeted run, 2 samples) | Scenario | 0.2.0 events/s | 0.3.0 events/s | Ratio | Statements per event, 0.2.0 / 0.3.0 | | --------------------------------------------------- | -------------: | -------------: | ----: | ----------------------------------: | | 1 publisher, 1 event per call | 1,543 | 1,573 | 1.02 | 4.07 / 4.07 | | 1 publisher, 100 events per call | 18,030 | 17,960 | 1.00 | 1.04 / 1.04 | | 32 publishers, 1 event per call | 6,235 | 6,111 | 0.98 | 4.02 / 4.02 | | 32 publishers, 100 events per call | 69,580 | 68,493 | 0.98 | 1.03 / 1.03 | | 8 publishers, 1 event per call, 1,000 distinct keys | 3,789 | 4,145 | 1.09 | 4.03 / 4.02 | All intervals include 1.0, so none of these is a change. A second targeted run (`2026-10-25T05-12-40-754Z`, candidate `13c2a13`, 2 samples per side) measured the remaining publish cells after the fixes: | Scenario | Ratio 0.3.0 / 0.2.0 | | --------------------------------------------------------------------- | ------------------: | | PostgreSQL, 8 publishers, 1 event per call | 1.00 | | PostgreSQL, 8 publishers, 100 events per call | 1.05 | | PostgreSQL, 4 processes x 8 publishers, 100 events per call (ceiling) | 0.97 | | PostgreSQL, 8 publishers, 1 event per call, 5 handlers | 1.07 | | PostgreSQL, 8 publishers, 100 events per call, 5 handlers | 1.02 | | PostgreSQL, 8 publishers, 1 event per call, 10 hot keys | 0.98 | | PostgreSQL, 8 publishers, 1 event per call, 1 hot key | 1.09 | | MongoDB, 8 publishers, 1 event per call | 1.02 | | MongoDB, 8 publishers, 100 events per call | 0.94 | | MongoDB, 8 publishers, 100 events per call, 5 handlers | 0.98 | | MongoDB, 8 publishers, 1 event per call, 10 hot keys | 1.06 | Every publish cell is now between 0.94 and 1.09 of 0.2.0, with two samples per side, so publishing is unchanged in 0.3.0. ### Dispatch on PostgreSQL (full run) | Scenario | 0.2.0 deliveries/s | 0.3.0 deliveries/s | Ratio | Samples | | ------------------------------------------------ | -----------------: | -----------------: | ----: | :-----: | | 1 process, `defaults` | 4,373 | 4,087 | 0.93 | 1 / 1 | | 1 process, `mid` | 11,140 | 10,890 | 0.98 | 1 / 1 | | 1 process, `tuned` | 15,613 | 14,255 | 0.98 | 3 / 3 | | 4 processes, `defaults` | 15,661 | 13,963 | 0.89 | 1 / 1 | | 4 processes, `mid` | 34,278 | 40,803 | 1.19 | 1 / 1 | | 4 processes, `tuned` | 53,844 | 56,831 | 1.06 | 1 / 1 | | 8 processes, `defaults` | 18,561 | 19,274 | 1.04 | 1 / 1 | | 8 processes, `mid` | 47,158 | 50,242 | 1.07 | 1 / 1 | | 8 processes, `tuned` | 59,814 | 52,865 | 0.88 | 1 / 1 | | Fan-out to 5 handlers, 1 process, `tuned` | 20,850 | 20,701 | 0.99 | 1 / 1 | | 1,000 distinct keys, 1 process, `tuned` | 4,200 | 4,133 | 0.98 | 1 / 1 | | 1,000 distinct keys, 4 processes, `tuned` | 4,453 | 7,931 | 1.78 | 1 / 1 | | 10 hot keys, 1 process, `tuned` | 2,022 | 1,955 | 0.97 | 1 / 1 | | Routed, 1,000 distinct keys, 5 ms ack, 1 process | 3,267 | 5,827 | 1.78 | 1 / 1 | | Routed, 10 hot keys, 5 ms ack, 1 process | 785 | 787 | 1.00 | 1 / 1 | The two 1.78 results are the ones this release is meant to help: with many keys and routes to one destination, per-key order now holds inside one handler instead of across several, so more keys can proceed at once. They are single samples, so the multiple is indicative; the direction is the same in two different cells. The 0.88 and 0.89 cells (8 processes `tuned`, 4 processes `defaults`) are as likely noise as a slowdown; one sample cannot tell, and a repeat run with three samples is the way to find out. Database counters, which do not depend on machine noise: WAL bytes per delivery were 8 to 11 % higher in four cells (1 process `tuned` with three samples, 4 processes `defaults` and `mid`, 8 processes `tuned`) and 5 to 8 % lower in five others. The benchmark gate flags the higher ones, which is why 0.3.0 is not described as faster per write. ### Dispatch on MongoDB (full run) | Scenario | 0.2.0 deliveries/s | 0.3.0 deliveries/s | Ratio | Samples | | ----------------------------------------- | -----------------: | -----------------: | ----: | :-----: | | 1 process, `defaults` | 1,149 | 1,161 | 1.01 | 1 / 1 | | 1 process, `tuned` | 6,431 | 6,228 | 0.98 | 3 / 3 | | 4 processes, `defaults` | 4,470 | 4,564 | 1.02 | 1 / 1 | | 4 processes, `tuned` | 21,771 | 22,452 | 1.03 | 1 / 1 | | Fan-out to 5 handlers, 1 process, `tuned` | 7,031 | 7,452 | 1.06 | 1 / 1 | | 1,000 distinct keys, 1 process, `tuned` | 2,089 | 2,131 | 1.02 | 1 / 1 | | 10 hot keys, 1 process, `tuned` | 347 | 349 | 1.01 | 1 / 1 | MongoDB is unchanged. MongoDB publish was also within noise (0.94 to 0.98 in five cells, one of them with three samples and an inconclusive interval). ### Mixed load | Scenario | Accepted events/s 0.2.0 to 0.3.0 | p99 latency 0.2.0 | p99 latency 0.3.0 | Samples | | ------------------------------- | -------------------------------: | ----------------: | ----------------: | :-----: | | PostgreSQL, 1,000/s, `defaults` | 1,000 / 1,000 | 9.9 ms | 10.4 ms | 1 / 1 | | PostgreSQL, 1,000/s, `tuned` | 1,000 / 1,000 | 9.0 ms | 14.4 ms | 1 / 1 | | PostgreSQL, 5,000/s, `tuned` | 4,133 / 4,032 | not meaningful | not meaningful | 2 / 2 | | MongoDB, 1,000/s, `tuned` | 1,000 / 1,000 | 474 ms | 377 ms | 1 / 1 | | MongoDB, 5,000/s, `tuned` | (saturated) | 7.5 s | 8.1 s | 1 / 1 | At 5,000 per second both versions are past saturation and the p99 is dominated by queueing. The same cell gave 79 ms against 10.6 s in the full run and 2.0 s against 104 ms in the targeted run, so its p99 ratios (0.05 to 134) say nothing about the code. Accepted throughput, 0.98, is the number to read. The 14.4 ms against 9.0 ms at 1,000 per second is one sample. ### Trust Same rule as above: one sample per side cannot separate a difference below about 10 % from noise, and where a ratio is outside that, the cell is called out. The exact counters (statements per publish, rows written, exactness) are the part that does not depend on the laptop. To check any cell on your hardware, run `pnpm bench:full --base v0.2.0 --only ` with more samples. ## Deep dives - [Write amplification in the delivery path](https://berkcansavur.github.io/relaybus/performance/write-amplification): why delivery rows were rewritten, lane leases, delete-on-complete and which guarantees are preserved. - [The publish path and prepared statements](https://berkcansavur.github.io/relaybus/performance/publish-path): why three cells were slower on the first 0.2.0 run, the round trips and plan times, and the fix. - [Tuning for your workload](https://berkcansavur.github.io/relaybus/tuning): the settings that follow from these numbers. ## Reproduce it ```sh pnpm install --frozen-lockfile pnpm bench:full --base v0.1.0 --candidate v0.2.0 --interleave ``` Requires Docker and Node 24. Output (`report.md`, `comparison.json`, `samples.jsonl`, `env.json`) goes to `benchmarks/results/runs//`. [`benchmarks/README.md`](https://github.com/berkcansavur/relaybus/blob/main/benchmarks/README.md) describes the harness, the cells and how verdicts are decided. --- Source: https://berkcansavur.github.io/relaybus/performance/write-amplification # Write amplification in the delivery path > This is the design deep-dive behind the 0.2.0 delivery path. It uses the numbers of the first measurement round; the > final, interleaved 0.2.0 against 0.1.0 comparison is on [Performance: 0.2.0 vs 0.1.0](https://berkcansavur.github.io/relaybus/performance). To move a > 0.1.0 installation, follow [Upgrading from 0.1 to 0.2](https://berkcansavur.github.io/relaybus/upgrading). This document explains why the first version of relaybus (0.1.0) spent most of its database time rewriting delivery rows, what Phase 1 changed, which guarantees that change must not weaken and how each one is preserved, and what it measured. - [The problem](#the-problem) - [What changed](#what-changed) - [Guarantees and how each one is preserved](#guarantees-and-how-each-one-is-preserved) - [Measurements](#measurements) - [MongoDB](#mongodb) - [Upgrading from 0.1.0](#upgrading-from-010) - [Trade-offs and operations](#trade-offs-and-operations) - [How to reproduce](#how-to-reproduce) ## The problem A delivery is the unit of work: one row per (event, handler). In 0.1.0 a delivery row went through three writes after it was inserted: 1. **claim**: `UPDATE ... SET status = 'processing', attempts = attempts + 1, lease_owner, lease_token, lease_expires_at` 2. **lease renewals** while the handler runs: `UPDATE ... SET lease_expires_at` 3. **completion**: `UPDATE ... SET status = 'completed', completed_at, lease cleared` The table also carried eight indexes (due time, ordering keys, status, handler, purge). Several of them contain `status` or the due time, and those are exactly the columns the three updates change. ### Dead tuples and HOT, in short PostgreSQL never changes a row in place. An `UPDATE` writes a **new version** of the row and leaves the old version behind as a **dead tuple** that autovacuum removes later. Normally the new version also needs a new entry in **every index** of the table, because each index points at the physical location of a version. There is one shortcut, a **HOT** (heap-only tuple) update. If the update changes no indexed column, and the new version fits on the same page, PostgreSQL chains the new version to the old one inside the page and adds **no index entries**: the indexes keep pointing at the first version of the chain. HOT updates are cheap and their dead versions are pruned from the page on the fly. A single update of an indexed column (here `status`) disables the shortcut for that update. 0.1.0 never got HOT updates, because every state change touched `status` or the due time. Measured on the baseline (`benchmarks/results/main`, one to eight dispatcher processes): **0% HOT updates, about 2.3 KB of WAL per delivery** and a steady stream of dead tuples and dead index entries. ### What one delivery leaves behind `benchmarks/pageinspect.ts` publishes one event with one handler, claims it and completes it, then reads the heap page and every index with `pageinspect` (PostgreSQL 17, autovacuum off so the evidence stays). Before (the complete output is in [`pageinspect-before.txt`](https://berkcansavur.github.io/relaybus/performance/pageinspect-before.txt)): ```text heap page of relaybus_deliveries (3 line pointers) lp xmin xmax ctid flags 1 775 776 (0,2) XMIN_COMMITTED|XMAX_COMMITTED 2 776 777 (0,3) XMIN_COMMITTED 3 777 0 (0,3) XMAX_INVALID indexes of relaybus_deliveries: 10 relaybus_deliveries_completed_idx entries=1 -> heap tids (0,3) relaybus_deliveries_due_idx entries=2 -> heap tids (0,1) (0,2) relaybus_deliveries_due_unkeyed_idx entries=2 -> heap tids (0,1) (0,2) relaybus_deliveries_event_handler_unique entries=3 -> heap tids (0,1) (0,2) (0,3) relaybus_deliveries_handler_created_idx entries=3 -> heap tids (0,1) (0,2) (0,3) relaybus_deliveries_pkey entries=3 -> heap tids (0,1) (0,2) (0,3) relaybus_deliveries_status_created_idx entries=3 -> heap tids (0,3) (0,1) (0,2) tuple counters: relaybus_deliveries inserted=1 updated=2 hot_updated=0 deleted=0 ``` Three heap tuples where each points at the next through `ctid` (none is `HEAP_ONLY` or `HEAP_HOT_UPDATED`: no HOT), two of them dead, and **17 index entries** for one delivery (6 at publish, 6 at claim, 5 at completion). Everything except the last tuple and its last entries is garbage that autovacuum has to find, and vacuuming the indexes costs more than vacuuming the heap. ![Life of one delivery row, before and after](https://berkcansavur.github.io/relaybus/performance/row-lifecycle.svg) ## What changed The idea comes from queue implementations that survived this problem (Oban, River, graphile-worker, pg-boss, PgQ): **do not write the row to say who has it, and delete the row when it is done.** ### Lane leases Every delivery gets a `lane`, a number from 0 to 255 computed from its ordering key (or from the event id when it has none). All deliveries of one key share a lane. Workers lease **lanes**, not rows. ![Lane lease model](https://berkcansavur.github.io/relaybus/performance/lane-leases.svg) ```sql CREATE TABLE relaybus_lanes ( lane smallint PRIMARY KEY CHECK (lane >= 0 AND lane < 256), epoch bigint NOT NULL DEFAULT 0, -- fencing token, +1 every time the lane changes hands lease_owner text, lease_expires_at timestamptz, in_flight text[] NOT NULL DEFAULT '{}' -- ids of the deliveries the owner has claimed ) WITH (fillfactor = 50); ``` - **Claim** (one short transaction): lock the lane rows this worker already holds, take free lanes that have due work (four, then eight, ... at a time, so a worker holds only as many lanes as its batch needs), read the due deliveries of those lanes, then write **one lane row per lane** to record the batch in `in_flight`. No delivery row is written and no row lock is taken on one. - **Renew**: `extendLease` updates the lane row, not the delivery. - **Complete**: delete the delivery rows. The dispatcher groups the completions of deliveries that finish within `dispatcher.completionBatchWindow` (2 ms by default) into one `completeMany` call, which is one statement (`DELETE ... USING` the fenced lane rows). - **Fail** (rare): `reschedule` and `bury` update the delivery (non-HOT, accepted) and remove it from `in_flight`. The lane table has no secondary index and 50% free space per page, so every update of a lane row is heap-only. Measured for one claim and completion: `relaybus_lanes inserted=256 updated=2 hot_updated=2` and, in the heap page, the chain `HOT_UPDATED -> HOT_UPDATED|HEAP_ONLY -> HEAP_ONLY` ([`pageinspect-after.txt`](https://berkcansavur.github.io/relaybus/performance/pageinspect-after.txt)): ```text heap page of relaybus_deliveries (1 line pointers) lp xmin xmax ctid flags 1 782 784 (0,1) XMIN_COMMITTED indexes of relaybus_deliveries: 6 -- 3 of them hold the row, the others stay empty relaybus_deliveries_event_handler_unique entries=1 -> heap tids (0,1) relaybus_deliveries_lane_due_idx entries=1 -> heap tids (0,1) relaybus_deliveries_pkey entries=1 -> heap tids (0,1) lane 218: the lane row this claim touched, versions of that row only lp xmin xmax ctid flags 18 779 783 (3,56) HOT_UPDATED|XMIN_COMMITTED|XMAX_COMMITTED 56 783 783 (3,57) HOT_UPDATED|HEAP_ONLY|XMIN_COMMITTED|XMAX_COMMITTED 57 783 784 (3,57) HEAP_ONLY|XMIN_COMMITTED tuple counters: relaybus_deliveries inserted=1 updated=0 hot_updated=0 deleted=1 relaybus_lanes inserted=256 updated=2 hot_updated=2 deleted=0 ``` One heap tuple (deleted by the completion, not rewritten) and **3 index entries** instead of 17. The lane row's updates added no index entry and are shared by every delivery of the batch. ### Delete on complete A completed delivery is deleted. `listDeliveries({ status: 'completed' })` and `statistics().completed` are empty unless the store keeps a **completion history** (`completionHistory: true`): an append-only table, one insert per completion, no updates. Events are still removed by retention (`retention.completed`) once they have no deliveries left. ### Schema without hot-path index churn Indexes on `deliveries` now only cover columns that no happy-path statement writes: | index | serves | | ----------------------------------------------------------------------------- | -------------------------------------------- | | `(event_id, handler)` unique, primary key | idempotent publish, fenced single-row writes | | `(lane, available_at, id) WHERE status = 'pending'` | claim, statistics | | `(lane, handler, key, sequence) WHERE key IS NOT NULL AND status = 'pending'` | per-key ordering check, blocked-key claim | | `(updated_at, id) WHERE status = 'dead'` | dead-letter purge | `status_created` and `handler_created` (listing) and the unkeyed and processing partial indexes are gone. The 0.1.0 indexes also made the planner choose `status_created` for `WHERE status = 'processing' AND id = ...` updates (357 to 525 buffers per single-row `UPDATE`, against 18 to 50 through the primary key). Fenced writes now always go through the primary key (`EXPLAIN` of a 50-delivery completion: 308 buffers, 0.23 ms). Both the lane table (`fillfactor = 50`) and `deliveries` (`autovacuum_vacuum_scale_factor = 0.02`) carry storage parameters set by `setup()`. ### Events remember their handlers Because the delivery row is gone after completion, `events.handlers` records every handler an event ever had a delivery for. A broker redelivery or a publish retry with the same event id adds deliveries only for handlers that are not listed, so a completed handler never runs twice (the RabbitMQ and Kafka inbox tests pin this). This costs one `text[]` on the event insert and no update. ## Guarantees and how each one is preserved | guarantee | how it is preserved | | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Atomic publish** | Unchanged: events and deliveries are inserted in the caller's transaction. | | **Never lose an event** | A delivery row exists from publish until its handler's completion is committed. A claim writes only to the lane. If the worker dies, the delivery is still a `pending` row, and the lane lease expires and is taken over, so it is redelivered. A completion deletes the row only after the handler returned (or in the handler's own transaction). | | **Per-handler retry and dead letter** | Each (event, handler) is its own row. `reschedule` and `bury` update that row exactly as before; a dead row is never claimed again; `requeue` moves it back with a fresh budget. | | **Per-key ordering in commit order** | Unchanged on the publish side (the key counter row is locked until commit). On the claim side a keyed delivery is picked only if no earlier delivery of its handler and key is still a `pending` row (a running one is one) and none of its key is in flight; and a key lives in one lane, owned by one worker at a time. | | **At-least-once** | An expired lane lease is taken over by the next claimer, so everything its holder had in flight runs again. Completion is the only thing that removes a row. | | **Effectively-once for transactional handlers** | `complete(lease, now, transaction)` takes `FOR KEY SHARE` on the lane row inside the handler's transaction and deletes the delivery there. A takeover locks the lane `FOR UPDATE`, which conflicts, so it skips the lane until the transaction ends. (The weakest conflicting lock is used on purpose: the owner's own claim and lease renewals are plain updates of the lane row and must not wait for every completion in progress.) Either the handler's writes and the deletion commit together, or the takeover finds the delivery and redelivers it into a transaction whose first completion rolled back. Both cannot commit. Tested in `lane-leases.integration.test.ts` ("does not let a completion and a takeover both win"). | | **Multi-process safety, fencing** | The lease token is `lane:epoch:claims`. Every state change checks that the lane still has that epoch, still lists the delivery in `in_flight`, and that the delivery's `claims` counter (bumped whenever a claim ends) still equals the token's, so the lease of an older attempt cannot touch a newer one. A takeover bumps the epoch, so all writes of the previous holder return `false`. Two claimers never see the same lane: the claim takes free lanes with `FOR UPDATE SKIP LOCKED`. | | **Attempts and poison messages** | The claim does not write an attempt. The attempt is recorded when it ends in failure (`reschedule`/`bury` add one), and when a lane is taken over each delivery that was in its `in_flight` list is charged one attempt (a crash is a lost attempt). A crash loop therefore still counts and ends in the dead letters (contract test "keeps counting attempts of a poison delivery until it is dead-lettered"). A graceful `release` charges nothing. | | **Graceful shutdown** | Aborted handlers are `release`d (no attempt spent), then the dispatcher calls `relinquish`, which hands back every lane that runs nothing, so other workers do not wait for the lease to expire. | | **Inbox deduplication** | `events.handlers` outlives the deliveries (see above). | ## Measurements Both columns come from **the same harness** (`benchmarks/`, including the exactness validation described in `benchmarks/README.md`), run on the same machine: the 0.1.0 baseline at commit `e222e0a` and Phase 1 at `perf/phase-1`. **Every one of the 51 drain/mixed runs of each side is VALID**: for every (event, handler) pair the handled set equals the published set (lost 0, unexpected 0), duplicates 0, per-key order violations 0, nothing left pending and no dead letters when the run finished. Read with care: the machine is a laptop (Apple M4, Docker Desktop VM) that was shared with other work during these runs (load average around 8), so absolute numbers vary by tens of percent between runs and the two columns were not taken back to back. A short idle re-check of the default configuration (20k events, `--quick`) gave 1 instance, defaults: 2,215 -> 3,439 deliveries/s, and 4 instances, tuned: 5,970 -> 28,254, which is why the full-run row "1 inst, defaults" (2,165 -> 1,586) should be treated as noise. Quote ratios, not absolute values. "Dead tuples after" is `n_dead_tup` of the deliveries table right after the window: before it counts the two dead versions per delivery that updates leave, after it counts the one deleted tuple per delivery that autovacuum has not reached yet (`autovacuum` was on; runs are short). **Headline.** PostgreSQL dispatch: up to **47,651** deliveries/s (8 dispatcher processes, tuned) against 3,958 before (12x); with one tuned process 2,631 -> 12,372 (4.7x). WAL per delivery **2,377-2,692 B -> 257-625 B** (about 5-9x less), deliveries updated **0** (was 2 per delivery), lane updates **100% HOT** (HOT updates on deliveries were 0% before and are not applicable now). MongoDB dispatch: **547-611 -> up to 10,422** deliveries/s (17x with 4 processes), oplog bytes per delivery **1,155 -> 198-266** and updates per delivery **1.0 -> 0.02-0.21**. Known regressions to track (not fixed in Phase 1): single-event publish on both stores is slower than 0.1.0 in these runs (PostgreSQL `P=1 batch=1` 1,350 -> 479 events/s in the full run, 867 -> 705 in the idle re-check; MongoDB large-batch publish about 0.2-0.4x), which is the cost of the extra `events.handlers` bookkeeping and of the claim-side work moving to publish time (lane hash, richer documents). They need a dedicated idle-machine investigation; the dispatch gains above dwarf them for any workload that processes what it publishes, but they are real. ### pg: dispatch (scenario 2, 200k preloaded events, no-op handler) | config | deliveries/s (before -> after) | WAL B/delivery | HOT upd % (deliveries / lanes) | dead tuples after | stmts/delivery | | -------------------------------------------------------- | ------------------------------ | ---------------- | ------------------------------ | ---------------------- | ---------------- | | 1 inst, defaults (conc 10, batch 10, pool 10, poll 1s) | 2,165 -> **1,586** (0.7x) | 2,377 -> **625** | 0.0 / 100 | 74,678 -> **62,904** | 1.21 -> **0.21** | | 1 inst, conc 10, batch 50 | 1,784 -> **3,146** (1.8x) | 2,345 -> **550** | 0.0 / 100 | 128,285 -> **0** | 1.25 -> **0.21** | | 1 inst, conc 50, batch 50, pool 20 | 2,696 -> **5,337** (2.0x) | 2,374 -> **384** | 0.0 / 100 | 123,533 -> **135,182** | 1.06 -> **0.04** | | 1 inst, conc 50, batch 200, pool 20 | 2,554 -> **4,263** (1.7x) | 2,372 -> **388** | 0.0 / 100 | 135,746 -> **160,014** | 1.06 -> **0.04** | | 1 inst, tuned (conc 200, batch 200, pool 40, poll 50ms) | 2,631 -> **12,372** (4.7x) | 2,517 -> **360** | 0.0 / 98 | 61,146 -> **142,111** | 1.02 -> **0.01** | | 1 inst, tuned+ (conc 400, batch 400, pool 64, poll 50ms) | 7,613 -> **14,825** (1.9x) | 2,432 -> **385** | 0.0 / 99 | 316,279 -> **110,415** | 1.01 -> **0.01** | | 4 inst, defaults (conc 10, batch 10, pool 10, poll 1s) | 2,411 -> **5,299** (2.2x) | 2,378 -> **461** | 0.0 / 100 | 161,297 -> **188,701** | 1.25 -> **0.21** | | 4 inst, conc 10, batch 50 | 4,053 -> **7,090** (1.7x) | 2,291 -> **458** | 0.0 / 100 | 337,389 -> **139,203** | 1.23 -> **0.21** | | 4 inst, conc 50, batch 50, pool 20 | 2,377 -> **5,426** (2.3x) | 2,529 -> **387** | 0.0 / 99 | 125,462 -> **169,648** | 1.06 -> **0.04** | | 4 inst, conc 50, batch 200, pool 20 | 6,727 -> **11,039** (1.6x) | 2,438 -> **368** | 0.0 / 99 | 313,921 -> **51,311** | 1.05 -> **0.04** | | 4 inst, tuned (conc 200, batch 200, pool 40, poll 50ms) | 7,338 -> **18,789** (2.6x) | 2,540 -> **257** | 0.0 / 94 | 308,922 -> **200,000** | 1.02 -> **0.01** | | 4 inst, tuned+ (conc 400, batch 400, pool 64, poll 50ms) | 2,486 -> **25,437** (10.2x) | 2,661 -> **286** | 0.0 / 97 | 141,181 -> **200,000** | 1.01 -> **0.01** | | 8 inst, defaults (conc 10, batch 10, pool 10, poll 1s) | 3,917 -> **7,019** (1.8x) | 2,361 -> **457** | 0.0 / 100 | 346,585 -> **156,411** | 1.24 -> **0.21** | | 8 inst, conc 10, batch 50 | 1,965 -> **4,390** (2.2x) | 2,444 -> **460** | 0.0 / 100 | 132,804 -> **177,232** | 1.27 -> **0.21** | | 8 inst, conc 50, batch 50, pool 20 | 2,517 -> **26,541** (10.5x) | 2,594 -> **387** | 0.0 / 98 | 178,156 -> **186,785** | 1.06 -> **0.04** | | 8 inst, conc 50, batch 200, pool 20 | 2,490 -> **26,811** (10.8x) | 2,605 -> **280** | 0.0 / 99 | 140,228 -> **200,000** | 1.06 -> **0.04** | | 8 inst, tuned (conc 200, batch 200, pool 40, poll 50ms) | 3,958 -> **47,651** (12.0x) | 2,601 -> **257** | 0.0 / 92 | 332,611 -> **200,000** | 1.02 -> **0.01** | | 8 inst, tuned+ (conc 400, batch 400, pool 64, poll 50ms) | 2,069 -> **30,332** (14.7x) | 2,692 -> **285** | 0.0 / 95 | 159,433 -> **200,000** | 1.01 -> **0.01** | ### pg: publish only (scenario 1) | config | events/s | publish call p50 / p99 ms | WAL B/event | | ---------------------------------------- | --------------------------- | ---------------------------------- | ------------------ | | P=1 batch=1 pool=default | 1,350 -> **479** (0.4x) | 0.40 / 0.9 -> **0.98 / 4.9** | 1,768 -> **1,485** | | P=1 batch=100 pool=default | 15,507 -> **14,733** (1.0x) | 5.28 / 11.8 -> **5.18 / 13.5** | 1,797 -> **1,487** | | P=8 batch=1 pool=default | 3,617 -> **2,604** (0.7x) | 0.94 / 2.3 -> **1.48 / 4.0** | 1,779 -> **1,507** | | P=8 batch=100 pool=default | 48,167 -> **29,593** (0.6x) | 9.99 / 25.7 -> **14.72 / 84.8** | 1,905 -> **1,545** | | P=32 batch=1 pool=default | 3,737 -> **2,339** (0.6x) | 1.13 / 2.9 -> **2.14 / 5.6** | 1,776 -> **1,503** | | P=32 batch=100 pool=default | 56,807 -> **42,987** (0.8x) | 11.48 / 29.5 -> **15.02 / 43.5** | 1,917 -> **1,551** | | P=8 batch=1 pool=10 | 3,343 -> **1,732** (0.5x) | 0.95 / 2.1 -> **2.22 / 6.4** | 1,779 -> **1,518** | | P=8 batch=100 pool=10 | 59,633 -> **47,720** (0.8x) | 9.13 / 21.1 -> **12.19 / 29.0** | 1,897 -> **1,538** | | P=32 batch=1 pool=34 | 5,135 -> **3,227** (0.6x) | 2.34 / 7.6 -> **3.55 / 15.9** | 1,773 -> **1,510** | | P=32 batch=100 pool=34 | 58,727 -> **33,987** (0.6x) | 35.69 / 93.7 -> **53.67 / 309.2** | 1,992 -> **1,585** | | ceiling 4 procs x 8 publishers batch=1 | 4,789 -> **3,046** (0.6x) | 2.38 / 9.1 -> **3.66 / 17.4** | 1,773 -> **1,506** | | ceiling 4 procs x 8 publishers batch=100 | 48,080 -> **35,167** (0.7x) | 43.12 / 142.3 -> **54.75 / 238.8** | 1,988 -> **1,593** | ### pg: mixed steady state (scenario 5), end-to-end commit-to-handler latency | config | offered/s | achieved/s (before -> after) | e2e p50 / p95 / p99 ms before | e2e p50 / p95 / p99 ms after | backlog max (before -> after) | | ---------------------------- | --------- | ---------------------------- | ------------------------------ | ------------------------------ | ----------------------------- | | 1000/s, defaults dispatchers | 1,000 | 1,000 -> **1,000** | 503.6 / 952.0 / 1,000.5 | 2,262.5 / 7,034.2 / 8,085.7 | 854 -> 4,731 | | 5000/s, defaults dispatchers | 5,000 | 3,383 -> **2,627** | 17,053.7 / 39,731.6 / 42,175.9 | 11,684.4 / 33,548.4 / 40,128.9 | 142,912 -> 91,608 | | 1000/s, tuned dispatchers | 1,000 | 981 -> **1,000** | 12.4 / 89.2 / 186.2 | 15.8 / 148.1 / 297.2 | 128 -> 197 | | 5000/s, tuned dispatchers | 5,000 | 1,897 -> **2,144** | 89.2 / 285.6 / 396.6 | 127.6 / 442.5 / 658.8 | 703 -> 627 | | 10000/s, tuned dispatchers | 10,000 | 2,149 -> **2,421** | 39.4 / 90.9 / 131.4 | 128.8 / 479.1 / 764.8 | 148 -> 906 | ### mongo: dispatch (scenario 2, 200k preloaded events, no-op handler) | config | deliveries/s (before -> after) | oplog B/delivery | oplog entries/delivery | journal B/delivery | ops/delivery | updates/delivery | | ------------------------------------------------------- | ------------------------------ | ---------------- | ---------------------- | ------------------ | -------------- | ---------------- | | 1 inst, defaults (conc 10, batch 10, pool 10, poll 1s) | 547 -> **1,069** (2.0x) | 1,157 -> **266** | - -> **0.31** | 1,445 -> **227** | 2.8 -> **1.0** | 1.01 -> **0.21** | | 1 inst, conc 50, batch 50, pool 20 | 597 -> **3,359** (5.6x) | 1,155 -> **219** | - -> **0.18** | 1,431 -> **142** | 2.2 -> **0.2** | 1.00 -> **0.08** | | 1 inst, tuned (conc 200, batch 200, pool 40, poll 50ms) | 609 -> **5,621** (9.2x) | 1,155 -> **198** | - -> **0.12** | 1,419 -> **108** | 2.1 -> **0.1** | 1.00 -> **0.02** | | 4 inst, defaults (conc 10, batch 10, pool 10, poll 1s) | 590 -> **2,901** (4.9x) | 1,156 -> **262** | - -> **0.30** | 1,424 -> **210** | 5.6 -> **1.0** | 1.00 -> **0.20** | | 4 inst, conc 50, batch 50, pool 20 | 452 -> **7,721** (17.1x) | 1,155 -> **223** | - -> **0.20** | 1,419 -> **146** | 5.1 -> **0.3** | 1.00 -> **0.09** | | 4 inst, tuned (conc 200, batch 200, pool 40, poll 50ms) | 611 -> **10,422** (17.1x) | 1,155 -> **199** | - -> **0.13** | 1,414 -> **110** | 5.0 -> **0.1** | 1.00 -> **0.03** | ### mongo: publish only (scenario 1) | config | events/s | publish call p50 / p99 ms | oplog B/event | | ---------------------------------------- | --------------------------- | ------------------------------------- | ------------------ | | P=1 batch=1 pool=default | 454 -> **360** (0.8x) | 1.12 / 3.0 -> **1.06 / 3.7** | 1,294 -> **1,351** | | P=1 batch=100 pool=default | 9,693 -> **2,453** (0.3x) | 7.48 / 16.1 -> **26.48 / 92.8** | 1,060 -> **1,109** | | P=8 batch=1 pool=default | 2,021 -> **830** (0.4x) | 2.01 / 4.8 -> **3.55 / 16.9** | 1,300 -> **1,348** | | P=8 batch=100 pool=default | 37,587 -> **13,200** (0.4x) | 13.20 / 43.5 -> **37.51 / 123.8** | 1,056 -> **1,107** | | P=32 batch=1 pool=default | 2,813 -> **1,839** (0.7x) | 4.93 / 14.3 -> **6.77 / 21.9** | 1,301 -> **1,353** | | P=32 batch=100 pool=default | 30,140 -> **5,293** (0.2x) | 65.49 / 248.4 -> **297.18 / 1,781.8** | 1,062 -> **1,098** | | ceiling 4 procs x 8 publishers batch=1 | 1,802 -> **1,083** (0.6x) | 6.91 / 29.2 -> **5.95 / 29.2** | 1,286 -> **1,342** | | ceiling 4 procs x 8 publishers batch=100 | 25,293 -> **17,633** (0.7x) | 72.34 / 318.6 -> **105.58 / 377.3** | 1,059 -> **1,107** | ### mongo: mixed steady state (scenario 5), end-to-end commit-to-handler latency | config | offered/s | achieved/s (before -> after) | e2e p50 / p95 / p99 ms before | e2e p50 / p95 / p99 ms after | backlog max (before -> after) | | ---------------------------- | --------- | ---------------------------- | ------------------------------ | ---------------------------- | ----------------------------- | | 1000/s, defaults dispatchers | 1,000 | 1,000 -> **1,000** | 15,906.3 / 27,222.2 / 27,769.4 | 1,581.3 / 3,141.8 / 3,720.9 | 27,750 -> 2,080 | | 1000/s, tuned dispatchers | 1,000 | 1,000 -> **1,000** | 20,398.8 / 31,920.2 / 32,887.4 | 168.5 / 513.7 / 1,083.4 | 33,814 -> 594 | | 5000/s, tuned dispatchers | 5,000 | 1,405 -> **1,658** | 26,160.0 / 45,218.2 / 45,218.2 | 3,019.3 / 6,895.6 / 8,414.0 | 63,928 -> 7,541 | ## MongoDB The MongoDB store had the same problem in another form: every claim was one conditional `findOneAndUpdate` per delivery (plus per-key reads), every completion one `updateOne` per delivery, and each of them rewrote indexed fields (`status`, `dueAt`, lease fields): about 3 oplog entries and 3 index rewrites per delivery and roughly 2.5 round trips. The 0.1.0 baseline tops out near 600 deliveries/s for one dispatcher, many times slower than PostgreSQL. Phase 1 gives MongoDB the same model, adapted to what MongoDB can do atomically: - **Lane documents.** A `lanes` collection holds 256 documents `{ _id: lane, epoch, owner, holder, leaseExpiresAt, inFlight, lost }`. Deliveries carry a `laneId` (FNV-1a hash of the ordering key or event id, modulo 256). The existing `lane: 'ready' | 'parked'` field of a delivery (claim-index membership) is unchanged; `laneId` is the lease partition. - **Claim without delivery writes.** A claim reads the candidates of the lanes it holds (and, a few at a time, free lanes with due work), checks the per-key rules, and records the batch with **one conditional `bulkWrite` on the lane documents**. No delivery document is written, so no indexed field of a delivery changes on the happy path. The lease token is `:`. - **Completion.** `completeMany` deletes the delivery documents of a batch in one transaction (and inserts the optional `completed_deliveries` history documents). There is no per-delivery update. - **Fencing without a lane write on completion.** MongoDB transactions use snapshot isolation, so two completions in one lane that both wrote the lane document would conflict with each other and transactional handlers would re-run. Instead a takeover moves the lane's `inFlight` ids into `lost` in the very write that bumps the epoch, and then charges each of them one attempt; a completion deletes the same delivery documents the charge updates. A completion and a takeover therefore write the same documents and exactly one of them commits; the other conflicts, retries, finds the new epoch and reports the lease as lost (tested, including a completion that started before a takeover). Concurrent completions of one lane do not conflict. - **Attempts.** Charged when an attempt ends (`reschedule`, `bury`) or when a lane is taken over (a crash). If the process dies between the epoch bump and the charge, the ids stay in `lost` and the next holder charges them first; a crash after the charge and before `lost` is cleared charges one extra attempt (a documented limitation). - **Restarted workers.** Lanes record the store instance (`holder`) that owns them, so a restarted process that reuses its `leaseOwner` string does not adopt its dead predecessor's live lanes; it takes them over after the lease expires. - **Failure handling.** A failure after the batch was recorded gives it back to the lane; parking of blocked followers is best effort (logged through the `logger` option); after recording, each keyed pick is re-checked against requeued predecessors and given back if it was overtaken. - **Events** carry `handlers` (a completed handler never runs again for a redelivered event) and `createdAt` (orphan event sweep). - **Upgrade.** `setup()` converts a 0.1.0 database in place, repeatably: `processing` deliveries become `pending` with their attempts kept, lease fields are removed, `laneId` and the events' `handlers` are backfilled in batches, obsolete indexes are dropped. The measured effect on oplog entries and bytes per delivery is in the MongoDB tables above. Keyed deliveries still pay per-key reads (an unfinished-predecessor check per head); the store parks long backlogs of one key out of the claim index so a deep queue behind one slow delivery costs nothing per claim. ## Upgrading from 0.1.0 The step-by-step guide is [Upgrading from 0.1 to 0.2](https://berkcansavur.github.io/relaybus/upgrading). The mechanics: MongoDB is upgraded by `setup()` as well (see above). PostgreSQL: `setup()` (run by `bus.start()` unless `setupStore: false`) upgrades a 0.1.0 database in place, inside the same advisory-lock transaction as a fresh install, and can be run any number of times. PostgreSQL, in order: 1. The upgrade runs only when the `deliveries.lease_token` column still exists (the marker of a 0.1.0 table). It adds `events.handlers` and fills it from the existing deliveries (completed ones included, so nothing completed under 0.1.0 runs again), adds `deliveries.lane` and fills it with the same hash new deliveries use, turns `processing` deliveries into `pending` (their attempt stays counted, exactly as a lost lease would count it), and drops `lease_owner`, `lease_token` and `lease_expires_at`. 2. The 0.1.0 indexes that no longer exist are dropped (`DROP INDEX IF EXISTS`), the new ones are created (`CREATE INDEX IF NOT EXISTS`), the 256 lane rows are seeded (`ON CONFLICT DO NOTHING`), and the storage parameters are set. 3. Deliveries that were `completed` stay as `completed` rows and keep being listed, counted and purged by `retention.completed` until they are gone. New completions are deleted. What to plan for: - **Stop every 0.1.0 worker first.** They would fail on the dropped columns. Running two versions side by side is not supported. - **One-off rewrite.** Step 1 updates every delivery row once. Purge old completed deliveries before upgrading a large table (`bus.purgeExpired()` with a short `retention.completed`), so there is less to rewrite. - **Your own migration tool.** `createPostgresSchemaSql()` renders the same statements as one script. Apply it in one transaction with the workers stopped, then start the new version with `setupStore: false`. - **Rollback.** There is no down migration: take a backup (or run the upgrade in a transaction you can roll back during rehearsal). Going back to 0.1.0 needs a restore. - **Test coverage.** The upgrade is tested against the verbatim 0.1.0 DDL with pending, processing, completed and dead keyed and unkeyed rows, run twice (`packages/postgres/test/lane-leases.integration.test.ts`). ## Trade-offs and operations What got worse, and what to do about it: - **Completed deliveries are not listable by default.** `listDeliveries({ status: 'completed' })` and `statistics().completed` are empty. Turn on `completionHistory` (PostgreSQL, MongoDB) or `retainCompleted` (in-memory store) if you need an audit trail. It costs one extra insert per completion, so keep it off on hot paths. - **Lane exclusivity.** While a worker holds a lane under a live lease, other workers do not see its pending deliveries, even if they are idle. A worker that crashes delays its lanes by the lease duration (`dispatcher.leaseDuration`, 30 s by default); a worker that stops gracefully gives them back at once. Lanes are taken per claim and handed back when they run nothing, so a backlog spreads over workers. A few hot ordering keys live in few lanes and therefore run on few workers, which is what ordering means anyway. - **Deleting still leaves dead tuples.** A delete leaves one dead tuple per delivery, not three, and no dead index entries from updates; autovacuum still has to reclaim them. The tables are created with `autovacuum_vacuum_scale_factor = 0.02`. For sustained thousands of deliveries per second, lower `autovacuum_vacuum_cost_delay` on the table or schedule a `VACUUM`, and alert on `n_dead_tup`. - **Retry and dead-letter updates are not HOT.** `reschedule` and `bury` update `available_at` and `status`, which are indexed. They are the rare path by design. `requeue` is an admin action. - **Admin listings are scans.** `listDeliveries` has no dedicated index any more (it used to have two). It orders by `created_at`, so on a large table filter by `status` or `handler`, or add the index you need. - **Failure accounting moved.** An attempt is charged when it ends or when a lane is taken over, not at claim. A worker that dies is charged at takeover, after the lease expired; between the crash and the takeover `listDeliveries` shows the deliveries as `processing`. - **Events outlive their deliveries.** Events (and their `handlers` list) are deleted by retention, not at completion; keep `retention.completed` longer than your longest broker redelivery window. After a purge a redelivery is a new event. - **A claim is more than one statement in the shallow-queue case.** When the held lanes do not fill the batch, the claim continues in a short transaction to take free lanes. With a deep backlog it is one statement. - **Fixed lane count.** 256 lanes are enough for any realistic number of workers; the count is stored in each delivery, so it is not a setting. - **Schema change.** Mixed 0.1.0 and new workers are not supported; upgrade is a stop-the-world migration (see above). Not part of Phase 1 (neither implemented nor measured here), candidates for a next phase: wake-up of idle dispatchers across processes with `LISTEN/NOTIFY` (a latency hint only, polling stays the correctness path), a single-statement publish that avoids the per-delivery foreign-key row lock on `events`, and a `UNLOGGED` or partitioned delivery table. ## How to reproduce The numbers on this page come from the first measurement round, taken with the earlier harness (`benchmarks/run.ts`, drain to completion, one sample per cell). That harness was replaced by the interleaved, fixed-window one that produced [the final comparison](https://berkcansavur.github.io/relaybus/performance). To reproduce current numbers: ```sh pnpm install --frozen-lockfile pnpm bench:quick # HEAD against main, about 8-10 minutes pnpm bench:full --base v0.1.0 --candidate v0.2.0 --interleave ``` [`benchmarks/README.md`](https://github.com/berkcansavur/relaybus/blob/main/benchmarks/README.md) describes the harness (every publisher and dispatcher is its own Node process, databases run in Testcontainers, retention is disabled so no purge interferes). The numbers above come from a MacBook (Apple M4, 10 cores, 16 GB) running Docker Desktop, with the Node workers and the databases sharing the same cores: read the ratios, not the absolute values. The old `pageinspect.ts` script is in the git history before the harness rework. --- Source: https://berkcansavur.github.io/relaybus/performance/publish-path # Publish path and dispatch statements: per-event cost, 0.1.0 against 0.2.0 > Part of [Performance: 0.2.0 vs 0.1.0](https://berkcansavur.github.io/relaybus/performance). Companion: [write amplification](https://berkcansavur.github.io/relaybus/performance/write-amplification). Three benchmark cells were slower on the 0.2.0 release branch than on 0.1.0 (`main`): `pg-pub-p1-b1` (−37 %), `pg-pub-p32-b1` (−8 %) and `pg-disp-1i-defaults` (−6 %). This page records why, measured with counters instead of throughput, and what changed. ## Method A PostgreSQL 17 container with the benchmark's settings ([`benchmarks/lib/pg-backend.ts`](https://github.com/berkcansavur/relaybus/blob/main/benchmarks/lib/pg-backend.ts): `synchronous_commit=on`, full-page writes, `pg_stat_statements` with `track_planning`), plus `pg_walinspect`. The probe publishes like the benchmark worker (`bus.transaction` + `bus.publish`, one handler, the benchmark's 160-byte payload) and counts: - round trips, by wrapping `client.query` of the pool; - statements parsed and planned by the server (`pg_stat_statements.plans`, named or unnamed protocol messages); - WAL bytes and records per relation (`pg_get_wal_block_info`), commits and WAL flushes (`pg_stat_wal`, `pg_stat_database`); - client-side time of each statement. To separate the database from relaybus' JavaScript, the captured statements were also replayed on a bare `pg` client, once unnamed (parsed on every call) and once as named prepared statements. Timings are medians of two runs of 10,000 publishes on an M4 laptop with Docker Desktop; the counts do not depend on the machine. ## Publish, one event per transaction (`pg-pub-p1-b1`) | per publish | 0.1.0 | 0.2.0 before | 0.2.0 now | | -------------------------------------- | ---------------------------- | ---------------------- | ---------------------- | | round trips in the transaction | 4 (BEGIN, 2 inserts, COMMIT) | 3 (BEGIN, 1, COMMIT) | 3 | | SQL text sent | 694 + 617 bytes | 2,726 bytes | none after the first | | statements parsed and planned | 2 | 1 (large) | 0 | | server time, plan + execute | 0.035 + 0.105 ms | 0.084 + 0.060 ms | 0 + 0.066 ms | | client time BEGIN / statement / COMMIT | 0.11 / 0.18 + 0.24 / 0.21 ms | 0.19 / 0.57 / 0.31 ms | 0.11 / 0.26 / 0.21 ms | | total per transaction | 0.75 ms (≈1,330 /s) | 1.15 ms (≈860 /s) | 0.59 ms (≈1,700 /s) | | WAL bytes per event | 1,740 | 1,475 | 1,475 | | rows written | events 1, deliveries 1 | events 1, deliveries 1 | events 1, deliveries 1 | | extra commit (trailing NOTIFY) | none | 0.07 to 0.15 | 0.04 to 0.07 | **Cause.** 0.2.0 publishes with one statement (`insertFreshEvents`: reserve key sequences, insert events, insert deliveries, notify) instead of two small inserts. The statement is 2.7 kB of SQL with five CTEs over four relations, and `pg` sent it unnamed, so the server parsed, analysed, rewrote and planned it on every publish. `pg_stat_statements` shows only 0.14 ms of planning and execution; the rest of the 0.57 ms is parse analysis of the text, which `pg_stat_statements` does not time. Replaying the exact statements on a bare client: | replay of one publish transaction | unnamed | prepared | | ---------------------------------- | ------- | -------- | | 0.1.0 statements | 0.75 ms | 0.61 ms | | 0.2.0 statement | 1.07 ms | 0.59 ms | | 0.2.0 without the key-sequence CTE | 0.99 ms | - | The BEGIN and COMMIT around the unnamed statement also took 0.08 to 0.10 ms longer; they return to 0.1.0's times once the statement is prepared (measured, the mechanism was not isolated). Ruled out: - **NOTIFY**: a build that never notifies published at 1.19 to 1.22 ms, the same as with notifications. - **WAL and rows**: 0.2.0 writes 15 % less WAL per event, because deliveries have 5 indexes instead of 9. - **Key sequences**: an unkeyed publish locks and writes no `key_sequences` row; dropping that CTE from the text saved only 0.08 ms. **Fix.** The store marks every statement whose text never changes, and `PgExecutor` sends it as a named prepared statement (name: `relaybus_` + 24 hex digits of its SHA-256). Each connection parses it once; later calls only bind parameters. After five executions PostgreSQL may switch to a generic plan; a design test forces generic plans for claim, takeover claim, completion, renewal and both publishes over 50,000 deliveries and checks that they use no sequential scan and stay far below the JIT threshold (highest estimate 10,608, threshold 100,000). ### Where the WAL of one published event goes | relation, record | 0.1.0 bytes | 0.2.0 bytes | | ---------------------------------------------- | ----------- | ----------- | | events heap insert | 449 | 473 | | events: pkey, created index | 198 | 198 | | events: row lock of the foreign key check | 54 | 54 | | deliveries heap insert | 179 | 179 | | deliveries: indexes | 7 = 717 | 3 = 301 | | speculative insert confirmations (ON CONFLICT) | 96 | 96 | | commit record | 34 | 34 | The events row is 24 bytes larger because an event remembers its handlers (`handlers text[]`). No lane, key sequence or completion row is written when publishing. ### Wake-up notifications A publishing transaction notifies only when it inserted a delivery due now, and a store sends at most one `NOTIFY` per 10 ms from its publishing statements; publishes in between leave one trailing `SELECT pg_notify` that the store sends after their commit, on a pool connection, outside the publishing transaction. It is not a round trip of the publish, but it is a transaction of its own with a commit record, at most 100 per second per store. Notifying from every publishing transaction instead would cost far more: PostgreSQL holds a database-wide lock from the commit of a notifying transaction until it is flushed, and with 32 publishers that measured 3,600 to 3,800 events/s against 6,500 coalesced. `wakeUp: 'poll'` removes notifications altogether. ## Publish with 32 connections (`pg-pub-p32-b1`) | 32 publishers, one event per transaction | 0.1.0 | 0.2.0 before | 0.2.0 now | | ---------------------------------------- | ------ | ------------ | --------- | | events per second | ≈5,150 | ≈4,760 | ≈6,500 | | WAL bytes per event | 1,744 | 1,472 | 1,472 | Same cause: with 32 connections the parse and planning of the 2.7 kB text on every publish keeps the server's CPUs busy. Coalesced notifications cost nothing measurable here (6,580 events/s without any notification against 6,470 with them). ## Dispatch, one process with default settings (`pg-disp-1i-defaults`) Concurrency 10, batch 10, 40,000 unkeyed deliveries after `VACUUM ANALYZE` and `CHECKPOINT`, 8 s. | per batch of 10 deliveries | 0.1.0 | 0.2.0 before | 0.2.0 now | | ---------------------------- | --------------------- | -------------------------------------- | -------------------- | | claim statements | 2 (of about 5) | 1 | 1 | | claim SQL text | about 2 kB each | 11.6 kB (21 kB when taking free lanes) | none after the first | | claim, client time | 1.40 ms each | 2.86 ms | 1.34 to 1.47 ms | | claim, server plan + execute | 0.77 ms | 1.19 ms | 0.41 ms (generic) | | completion statements | 10 (one per delivery) | 1 | 1 | | completion, client time | 0.83 ms each | 1.05 ms | 0.93 to 0.97 ms | | commits per delivery | 1.13 | 0.20 | 0.20 | | deliveries per second | 2,800 to 3,400 | 2,200 to 2,370 | 3,390 to 3,620 | Same cause again: the claim statement is long, and its parse and planning took more time than its execution. Prepared, a claim costs less than 0.1.0's two claims per ten deliveries. **WAL after a checkpoint.** The benchmark gate also flags `walBytesPerUnit` of this cell. Per delivery, 0.2.0 writes fewer records than 0.1.0 (about 8 against 17), but the completion inserts an `event_completions` row whose foreign key check locks the event row. Locking dirties the events page, and the first change of a page after a checkpoint writes the whole page to WAL: 0.11 to 0.17 full-page images per delivery (about 1 kB), against 0.05 for 0.1.0, whose deliveries were updated in place on fewer pages. It fades as pages are touched again within a checkpoint cycle and does not affect throughput in these measurements; it was left as is. ## Guards [`perf-budgets.json`](https://github.com/berkcansavur/relaybus/blob/main/perf-budgets.json) holds the measured values; a budget may only go down. | budget | value | 0.1.0 | | ---------------------------------------------------- | ------ | ------ | | `postgres.publish.unkeyedSingle.statements` | 3 | 4 | | `postgres.publish.unkeyedSingle.parsedStatements` | 0 | 2 | | `postgres.publish.keyedSingle.parsedStatements` | 0 | 2 to 3 | | `postgres.dispatch.parsedStatementsPerBatch` | 0 | 2+ | | `postgres.wal.publishSingle.bytesPerEvent` | 1,144 | - | | `postgres.explain.generic.largeTableSequentialScans` | 0 | - | | `postgres.explain.generic.maximumEstimatedCost` | 25,000 | - | The WAL test publishes 200 events of the test suite's envelope, one per transaction and more than 10 ms apart, on a server without full-page writes; with the benchmark's event 0.1.0 writes 1,740 bytes and 0.2.0 1,475. ## Not changed - **Adapters** (Prisma, TypeORM, Drizzle, Knex, Kysely) send every statement unprepared: their APIs do not name statements. Their publishes and claims still pay the parsing; see the [tuning guide](https://berkcansavur.github.io/relaybus/tuning) for running dispatchers on a `pg` pool. - **PgBouncer before 1.21 in transaction mode** does not keep prepared statements. `preparedStatements: false` avoids them; left on, the first statement that lands on another server session fails, after which the executor stops preparing and logs a warning. --- Source: https://berkcansavur.github.io/relaybus/faq # FAQ ## How do I publish events reliably in NestJS? Write the event to your database in the same transaction as your data, then deliver it after commit. With relaybus, register `RelayBusModule.forRoot({ store: new PostgresEventStore({ pool }) })`, call `bus.publish(event)` inside `bus.transaction(...)`, and declare handlers with `@RelayBusConsumer(EventClass)`. If the transaction commits, every handler is called at least once, retried with backoff until it succeeds or is dead-lettered. If it rolls back, the event never existed. See [Getting started](https://berkcansavur.github.io/relaybus/getting-started). ## How to implement the transactional outbox pattern in NestJS? Store each event in an outbox table inside the business transaction, and run a relay that claims due rows, delivers them and marks them done. relaybus implements this for you: `publish` inserts the event and one delivery per handler into your transaction, a dispatcher claims deliveries with leases, and failures are rescheduled with exponential backoff and jitter. You do not write the table, the polling loop or the retry logic. See [Transactional outbox](https://berkcansavur.github.io/relaybus/concepts/outbox). ## How to avoid losing events between the database and Kafka? Do not write to the database and Kafka separately. Write the event to the outbox in your database transaction and let a Kafka route forward it after commit. With `@relaybus/kafka`, the route is retried until the broker acknowledges (idempotent producer, `acks=-1`), and the consuming service commits the Kafka offset only after storing the event in its inbox, so duplicates are stored once. See [Kafka between services](https://berkcansavur.github.io/relaybus/recipes/kafka-between-services). ## What is the dual-write problem? A dual write updates two systems (for example a database and a message broker) without a shared transaction. If one write fails or the process crashes between them, the systems disagree: an event is lost, or an event announces a change that rolled back. The transactional outbox removes the second write from the request path. ## Does relaybus provide exactly-once delivery? No. Delivery to each handler is at-least-once. A handler subscribed with `transactional: true` that writes through its transaction takes effect exactly once for its database writes, because the writes and the delivery completion commit together. Side effects outside your database, such as HTTP calls, can repeat; send an idempotency key. See [Guarantees](https://berkcansavur.github.io/relaybus/guarantees). ## How do I make event handlers idempotent? Deduplicate on `event.id`, or use `transactional: true` and write through `context.transaction` so the effect and the completion commit together. Publishing with a deterministic `id` makes publishing itself idempotent: `bus.publish(event, { id: 'order-123' })`. ## How do I retry failed event handlers in NestJS? Set `retry` on `@RelayBusConsumer`: `@RelayBusConsumer(OrderPlaced, { retry: { attempts: 5, initialDelay: '2s', maxDelay: '5m' } })`. The defaults are 10 attempts with capped exponential backoff and full jitter. Throw `NonRetryableError` to skip the remaining attempts. See [Retries and backoff](https://berkcansavur.github.io/relaybus/concepts/retries). ## How do I inspect and requeue failed events? Use `bus.listDeliveries({ status: 'dead' })` to list dead letters with their last error, and `bus.requeue(ids)` to run them again once the cause is fixed. See [Dead letters](https://berkcansavur.github.io/relaybus/concepts/dead-letters). ## How do I keep events in order? Pass a `key` when publishing. Events with the same key are handled one at a time per handler, in commit order. There is no global ordering. See [Ordering keys](https://berkcansavur.github.io/relaybus/concepts/ordering-keys). ## Which databases and brokers does relaybus support? PostgreSQL 13 or newer through `@relaybus/postgres` (pg, Prisma, TypeORM, Drizzle, Knex and Kysely), MongoDB 6 or newer on a replica set through `@relaybus/mongodb` (driver or Mongoose sessions), Kafka through `@relaybus/kafka` and RabbitMQ through `@relaybus/rabbitmq`. You can bring your own store by implementing `EventStore` and running the contract suite from `@relaybus/testing`. ## Can I use relaybus without NestJS? Yes. `@relaybus/core` works with any Node.js application: create `new RelayBus({ store })`, call `bus.subscribe(...)`, `await bus.start()` and publish inside `bus.transaction(...)`. See [@relaybus/core](https://berkcansavur.github.io/relaybus/packages/core). ## How is relaybus different from @nestjs/event-emitter, BullMQ or pg-boss? `@nestjs/event-emitter` is in-process and does not persist events. BullMQ stores jobs in Redis, which cannot join your SQL or MongoDB transaction. pg-boss is a PostgreSQL job queue that can enqueue in your transaction; relaybus is an event bus with one delivery per handler, ordering keys and Kafka and RabbitMQ transports. See [the comparison](https://berkcansavur.github.io/relaybus/comparison). ## How do I test code that publishes events? Use `InMemoryEventStore`, disable the dispatcher, call `bus.drain()` to run due deliveries and `FakeClock` to advance time past retry delays. See [Testing with FakeClock](https://berkcansavur.github.io/relaybus/recipes/testing-with-fakeclock). ## Which Node.js and NestJS versions are supported? Node.js 22.12 or newer and NestJS 11 or 12. --- Source: https://berkcansavur.github.io/relaybus/releases # Releases - [0.5.0](#_0-5-0-2026-10-27): `route(produce(Catalog, { events }), settings)` routes several catalog events with shared settings; wire names declared on catalogs and routes reach the codec; transactions at an isolation level with automatic rerun of serialization failures - [0.4.1](#_0-4-1-2026-10-26): PostgreSQL hands out payloads exactly as published, Prisma 5 support, jsonEnvelope fixes, a codec contract for any test framework - [0.4.0](#_0-4-0-2026-10-25): Prisma 7 on a real database, wake-up at the caller's commit, the Confluent Kafka client, codec contract helpers, record keys, legacy flat envelopes, an application-owned producer - [0.3.1](#_0-3-1-2026-10-25): internal readability refactor, same behaviour - [0.3.0](#_0-3-0-2026-10-25): code quality, per-key order across event types, codecs, transactions in NestJS, bounded shutdown, ESLint rules, OpenTelemetry - [0.2.0](#_0-2-0-2026-10-24): high-throughput delivery, event catalogs, decoding of foreign messages - [0.1.x](#_0-1-x): first public release - [How versioning works](#how-versioning-works) ## 0.5.0 (2026-10-27) A minor release with new API and nothing required when upgrading from 0.4.x: no export removed or renamed, every option additive, no database or default wire-format change, and per-event routes and `jsonEnvelope` `wireTypes` work as before. See [Upgrading from 0.4.1 to 0.5.0](https://berkcansavur.github.io/relaybus/upgrading#upgrading-from-0-4-1-to-0-5-0). - **Catalog routes.** `route(produce(Catalog, { events }), { transport, destination, retry, timeout })` in `routes` turns every selected event into a route with the shared settings, where a service publishing several catalog events to one topic used to write one route per event and repeat the destination, retry and timeout. Without `destination`, each event goes to its own event type, as a plain route does; without `events`, the whole catalog is routed. The routes it creates are ordinary routes: one handler `relaybus.transport::` per destination (the per-key order and the ids of stored deliveries are the same as with routes written by hand), counted by `produce(...)` boot checks and by `RelayBusModule.forFeature({ produces })`. See [Routing a catalog production](https://berkcansavur.github.io/relaybus/concepts/event-catalogs#routing-a-catalog-production). - **Wire names next to the catalog and the route.** `defineCatalog(events, { wireTypes })` and `route(..., { wireTypes })` name the types a wire format uses, by catalog event name (checked by the compiler; the route's names win). The bus hands the names of the events a transport sends to it when the bus is created: the new optional `Transport.declareWireTypes` is implemented by the Kafka and RabbitMQ transports, which pass them to a `jsonEnvelope` codec through its new `withWireTypes`. The codec then writes and reads those events exactly as with its own `wireTypes`; a contract test on both transports checks that the bytes are identical. `catalogWireTypes(catalog)` lists a catalog's names for a codec on the consuming side. See [Wire names](https://berkcansavur.github.io/relaybus/concepts/event-catalogs#wire-names-next-to-the-catalog-and-the-route). - **Boot checks with their own codes.** An event routed twice to one destination through a catalog route (`RELAYBUS_ROUTE_DUPLICATE_EVENT`), a wire name for an unknown event (`RELAYBUS_CATALOG_UNKNOWN_EVENT`), a clash of wire names on a transport or with the codec's own (`RELAYBUS_WIRE_TYPE_COLLISION`) and wire names for a transport or codec that cannot map type names (`RELAYBUS_WIRE_TYPES_UNSUPPORTED`) fail when the bus is created, before any transport connects. Every transport checks its names first and all apply them last, so a refused configuration changes no transport; a transport that two buses use, or that already started, refuses wire names. All four extend `ConfigurationError`. Two plain routes to one destination keep failing with the plain `ConfigurationError`. - Additive API: core `route`, `catalogWireTypes`, `CatalogRoute`, `CatalogRouteOptions`, `RouteDefinition`, `DuplicateRouteEventError`, `UnknownCatalogEventError`, `WireTypeCollisionError`, `WireTypesUnsupportedError`, `DefineCatalogOptions.wireTypes`, `Transport.declareWireTypes` (checks the names and returns the function that applies them); `RelayBusOptions.routes` accepts `RouteDefinition` (a plain route or a catalog route); Kafka and RabbitMQ `KafkaCodec.withWireTypes` / `RabbitMqCodec.withWireTypes` and `declareWireTypes` on both transports; `InMemoryTransport.declaredWireTypes` in `@relaybus/testing`; `mergeWireTypes` and `withDeclaredWireTypes` in `@relaybus/core/internal`. - **Isolation levels with automatic rerun.** `bus.transaction(work, { isolationLevel, maxAttempts })` and `@RelayBusTransaction({ isolationLevel, maxAttempts })` run a transaction at `'read committed'`, `'repeatable read'` or `'serializable'`. At the last two, a serialization failure (SQLSTATE `40001`, Prisma `P2034`) rolls the attempt back and runs `work` again in a new transaction, up to `maxAttempts` times (default 3), and the last failure is thrown unchanged. This lets PostgreSQL's optimistic SERIALIZABLE replace `pg_advisory_xact_lock` or `SELECT ... FOR UPDATE` for write-skew checks such as "count the active rows, check a limit, insert". `work` may run more than once, so keep external calls out of it; events of a failed attempt are rolled back with it, and `onPublished` and the dispatcher wake-up wait for the final commit. `PgExecutor` sets the level in the `BEGIN` round trip (no extra statement), TypeORM through its own isolation-level transaction, every other executor with `SET TRANSACTION ISOLATION LEVEL` as the first statement; MongoDB and the in-memory store reject a level with a `ConfigurationError`. A decorated method that asks for a level joins a running transaction only when relaybus opened it at the same level, else it throws `TransactionIsolationConflictError` (`RELAYBUS_TRANSACTION_ISOLATION_CONFLICT`). Without `isolationLevel` a transaction runs exactly as in 0.4.1: same statements, same round trips, no rerun. See [Serializable transactions](https://berkcansavur.github.io/relaybus/recipes/serializable-transactions). - Additive API: core `TransactionOptions`, `TransactionIsolationLevel`, `TRANSACTION_ISOLATION_LEVELS`, `TransactionIsolationConflictError`, `RelayBus.currentIsolationLevel()`, `EventStoreTransactionOptions.isolationLevel`, and the optional `EventStore.isolationLevels` and `EventStore.isSerializationFailure`; `@RelayBusTransaction(options)`; postgres `PostgresIsolatedTransactionOptions` and the optional `PostgresExecutor.transactionWithIsolation` and `PostgresExecutor.isSerializationFailure`; a transaction-isolation section in the store contract of `@relaybus/testing`. - Benchmark against 0.4.1 (quick gate, interleaved; publish 6 pairs, dispatch 3 pairs per cell): publish 0.92–1.01x (batch 1 1.00x, batch 100 1.01x, batch 100 with 5 handlers 0.92x, every interval spans 1.0), dispatch 0.96–1.02x (PostgreSQL 1 instance 1.02x, 4 instances 1.01x, 10 hot keys 0.96x), so speed is unchanged; statements per event and per delivery are identical. MongoDB was not measured for this release: the harness could not load the 0.4.1 side's MongoDB template within its transaction budget. See [Performance](https://berkcansavur.github.io/relaybus/performance#_0-5-0-vs-0-4-1). ## 0.4.1 (2026-10-26) A patch release: no export removed or renamed, every option additive. PostgreSQL users get one column conversion that `setup()` applies; see [Upgrading from 0.4.0 to 0.4.1](https://berkcansavur.github.io/relaybus/upgrading#upgrading-from-0-4-0-to-0-4-1) for the DDL when you run your own migrations. - **PostgreSQL keeps the payload as published.** `payload` and `headers` were `jsonb`, which re-orders object keys, and the insert also went through `jsonb_to_recordset`, so an event read back serialized differently from what the producer published. Both columns are now `json` and the event insert uses `json_to_recordset` end to end, so the transport and the handlers get the producer's exact text, key order included. `setup()` converts existing tables in place (idempotent, one table rewrite); rows stored before keep the order `jsonb` gave them. The schema marker stays the one 0.4.0 writes, so 0.4.0 and 0.4.1 instances can run side by side and roll back with the default `setupStore: true`: a 0.4.0 `setup()` on a converted database runs nothing. The other batch parameters (deliveries, key counts, id lists) stay on `jsonb_to_recordset`. - **Payload fidelity is part of the store contract.** Every store (PostgreSQL through `pg`, Kysely, Knex, TypeORM, Drizzle and Prisma, MongoDB, in-memory) must hand back a payload with unsorted keys at every depth, unicode, `1e21`, `5e-324`, the largest safe integers and empty containers exactly as `JSON.stringify` wrote it, for single and batched appends and for a handler added later. A bus-level test checks the same through `publish` and `publishAll`. New tests also show that every built-in Kafka and RabbitMQ codec and both transports write and read payloads byte for byte; they did already. - **Prisma 5.** The optional `@prisma/client` peer is `^5.0.0 || ^6.0.0 || ^7.0.0` (0.4.0: `^6 || ^7`, which made npm refuse the install next to Prisma 5 or add a second client). `createPrismaExecutor` works unchanged on Prisma 5.22; a `prisma compat (prisma 5)` CI job runs the Prisma tests on it, and CI checks that installing `@relaybus/postgres` into an app on Prisma 5.22 leaves its `@prisma/client` and `prisma generate` untouched. - **jsonEnvelope keeps every top-level field type.** New `otherFields: { payloadField }` keeps the unknown top-level fields that are not read as event headers (numbers, booleans, null, objects, arrays, and text when headers do not live at the top level) unchanged under that payload field (an object payload always gets it, `{}` when there is nothing to keep), where the event's schema validates them; writing spreads them back over the top level, so the values survive a round trip, and the bytes do too when `bodyOrder` lists the fields in wire order. Without it, decode leaves them out as before. - **jsonEnvelope `bodyOrder` covers spread fields.** Event headers spread by `headers: 'top-level-fields'` (and fields kept by `otherFields`) now take their `bodyOrder` position; before they always came last. - **jsonEnvelope `wireTypes`.** Maps event types to the names a format uses on the wire, so one route carries many types whose wire names are not relaybus event types, without a `fixed` type per transport. The event type grammar is unchanged; it already accepts snake_case such as `transfer_auto_dispatched`. - **The codec contract runs under any test framework.** `@relaybus/testing/codec-contract` imports no test framework: `assertKafkaCodecCase` / `assertRabbitMqCodecCase` throw a `CodecContractError` (code `RELAYBUS_TESTING_CODEC_CONTRACT`), and `registerKafkaCodecContract` / `registerRabbitMqCodecContract` register tests with any `describe` and `it`. `@relaybus/testing/jest` registers with Jest's globals. `vitest` is an optional peer now; `kafkaCodecContract` and `rabbitMqCodecContract` from the main entry work with Vitest as before. Compared byte for byte, the decoded payload must also keep the expected key order. - Additive API: `jsonEnvelope` options `otherFields` and `wireTypes`; `@relaybus/testing` `assertKafkaCodecCase`, `assertRabbitMqCodecCase`, `registerKafkaCodecContract`, `registerRabbitMqCodecContract`, `CodecContractError`, `TestFramework`, and the `codec-contract` and `jest` entry points. - Benchmark against 0.4.0 (quick gate, 3 interleaved pairs per cell): publish 0.98–1.10x (batch 1 0.98x, batch 100 1.04x, batch 100 with 5 handlers 1.10x, MongoDB 0.99x), dispatch 1.02–1.03x (PostgreSQL 1 instance 1.02x, 10 hot keys 1.02x, MongoDB 1.03x), so speed is unchanged. WAL bytes per event or delivery are 3–11% lower and `lanes` updates per delivery 0.88x on the PostgreSQL dispatch cell. The 4-instance PostgreSQL dispatch cell was not valid on either side (its template drains before the measurement window ends); that is a harness sizing issue, not a regression. ## 0.4.0 (2026-10-25) Not breaking: the public API snapshot of every package changes by additions only (no export removed or renamed, checked with `git diff` of the `public-api` snapshots against 0.3.1), and nothing in the database, the SQL or the default wire format changes. Nothing to do when upgrading from 0.3; see [Upgrading from 0.3 to 0.4](https://berkcansavur.github.io/relaybus/upgrading#upgrading-from-0-3-to-0-4) for the optional features. - Prisma 7 runs on a real database. The Prisma adapter now passes the full event store contract against PostgreSQL with Prisma Client 7.10 on `@prisma/adapter-pg` (6.19 is covered in CI too). This found one bug: Prisma raw queries reject a `void` column, so `setup()` and the trailing notification failed; both statements now return a boolean column. `@prisma/client` is an optional peer `^6 || ^7`. - `PostgresEventStore` accepts `pool` next to `executor`. Statements outside a transaction (claims, completions, renewals) run on it as prepared statements (a warm dispatch batch parses 0 statements instead of 2 for Prisma, Kysely and Drizzle), and the store listens on it. - Wake-up at the caller's commit. With a `pool`, an event published in your ORM transaction is dispatched right after that transaction commits (handler start a median under 10 ms after a commit held back 200 ms). A publish in a transaction relaybus does not own wakes the dispatcher 25, 100 and 250 ms after the latest publish instead of once at 25 ms. New `bus.wake()` for stores without notifications. - Kafka: `client: 'confluent'` runs `@confluentinc/kafka-javascript` (its KafkaJS-compatible API), now the recommended client, tested on a real broker next to `kafkajs`. The default stays `'kafkajs'`. The peer range is `^1.11.0`. - Kafka: `producer` publishes and dead-letters through an already-connected producer the application owns (kafkajs or Confluent). The transport never connects or disconnects it. With it, `kafka` is optional and the transport publishes only; `kafka` and `groupId` as well make it consume with its own consumer. - Kafka `jsonEnvelope`: `recordKey: 'event-key' | 'none' | (event) => string | null` (default `'event-key'`, unchanged) and `contentTypeHeader`. Only the wire key changes; the outbox order of one event key stays. - `jsonEnvelope` (Kafka and RabbitMQ) describes more in-house formats: `occurredAt` as `'epoch-millis'` or `'epoch-seconds'`, `extra` body values and headers (constants or functions of the event, written only), `bodyOrder` for formats compared byte for byte, an id that falls back to the transport's own (`whenMissing: 'record-position'`) and `headers: 'top-level-fields'`. Together they write and read legacy flat envelopes (one topic, many types, no key, no headers) byte for byte. - `@relaybus/testing`: `kafkaCodecContract` and `rabbitMqCodecContract` (and the pure `kafkaCodecMismatches`, `rabbitMqCodecMismatches`) check a codec against captured messages in both directions. The transports' own decode path is exported as `decodeKafkaRecord` and `decodeRabbitMqMessage`. - Peer and test dependencies moved to the latest versions: `@opentelemetry/api` 1.10 with the OpenTelemetry SDK 3, `@nestjs/*` 12.2. - Additive API: Kafka `KafkaInjectedProducer`, `KafkaRecordKey`, `KafkaRecordOptions`, `decodeKafkaRecord`; RabbitMQ `decodeRabbitMqMessage`; testing the codec contract functions and their option types; `RelayBus.wake()`; `KafkaTransportOptions.kafka` is now optional. - Benchmark against 0.3.1 (quick run, 3 interleaved pairs per cell): publish 0.98–1.03x, dispatch 0.99–1.06x, statements per delivery identical. See [Performance](https://berkcansavur.github.io/relaybus/performance#_0-4-0-vs-0-3-1). - Known limits: on decode, unknown top-level fields are kept only when they are strings (0.4.1 adds `otherFields`); there is no consumer injection (only a producer can be passed in); RabbitMQ has no routing-key override. ## 0.3.1 (2026-10-25) A readability release: no behaviour, API, SQL or wire-format change. Nothing to do when upgrading from 0.3.0. - Every source file is now at most 300 code lines and every function at most 40 (enforced by lint). The largest files shrank from 794 to 294 lines (`relay-bus.ts`) and from 772 to 173 lines (`dispatcher.ts`). - Shared skeletons moved into abstract base classes: one lease settlement flow for the PostgreSQL, MongoDB and in-memory stores, one claim flow for PostgreSQL and MongoDB, and one subscription lifecycle for Kafka and RabbitMQ. Value objects replace repeated primitives with rules (durations, backoff, lease identity, composite keys, bounded integers). - Proof of no behaviour change: the public API snapshot, the SQL snapshot of every statement and the performance budgets are unchanged; 2,798 tests with 100% coverage. - Additive type change: `KafkaTransport` and `RabbitMqTransport` now extend the internal `SubscribingTransport` base class. - Benchmark against 0.3.0 (quick gate, 3 interleaved pairs per cell): publish 0.99–1.02x, dispatch 0.98–1.04x, every interval spans 1.0, so speed is unchanged. The 4-instance PostgreSQL dispatch cell was not valid on either side (its template drains before the measurement window ends); that is a harness sizing issue, not a regression. ## 0.3.0 (2026-10-25) Follow [Upgrading from 0.2 to 0.3](https://berkcansavur.github.io/relaybus/upgrading). Nothing has to be migrated in the database; read [Breaking changes](#breaking-changes) first. What stays stable between releases is described in [Stability](https://berkcansavur.github.io/relaybus/stability), what is tested in [Compatibility](https://berkcansavur.github.io/relaybus/compatibility). ### What is new | Area | Change | | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Code quality | The code now follows a strict set of limits: strict `tsconfig` flags (`noUnusedLocals`, `noImplicitReturns`, `noPropertyAccessFromIndexSignature` and more), lint rules for function size (60 lines), complexity (10), depth (3), at most two positional parameters, one class per file and a `[ClassName](methodName)` prefix on every log line. The two largest files, the PostgreSQL and MongoDB stores, are split into modules per concern with byte-identical SQL and unchanged filters. 2,773 tests, 100 % coverage unchanged. | | Errors | Every error class carries a stable `code` (`RELAYBUS_LEASE_LOST`, ...), reported on dead letters, retries and observer events and listed on [Error codes](https://berkcansavur.github.io/relaybus/concepts/error-codes). Remaining bare throws became coded classes. Kafka and RabbitMQ dead letters carry the error name and code. | | API surface | A snapshot test pins every export of every package, so a change of the public API is always deliberate. | | Ordering | Per-key order now holds across event types routed to one destination: all routes to one destination are one handler. [Ordering keys](https://berkcansavur.github.io/relaybus/concepts/ordering-keys). | | Codecs | Kafka and RabbitMQ take a symmetric `codec` (`cloudEventsBinary`, `cloudEventsStructured`, `jsonEnvelope`, or your own `{ encode, decode }`) for the wire format both ways. The default stays byte for byte the same. [Guide](https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services). | | Typed ordering key | `defineEvent(type, { schema, key: (payload) => payload.orderId })` derives the key from the payload; a wrong field fails to compile. | | Transactions in NestJS | `@RelayBusTransaction()` opens a transaction around a method, `RelayBusTransactionHost` hands it to repositories, and every transaction is held to `transactionTimeout` (default `'5s'`). [Guide](https://berkcansavur.github.io/relaybus/recipes/transactions-in-nestjs). | | Decorators | `@RelayBusConsumer(Event)` is the method decorator (default style) and `@RelayBusCatalogConsumer(Catalog)` the optional class decorator. `RelayBusModule.forFeature({ consumes, produces })` fails the boot when a module lacks a consumer or a route. [Event catalogs](https://berkcansavur.github.io/relaybus/concepts/event-catalogs). | | Bounded shutdown | `bus.stop({ timeout })` covers the whole shutdown, force-closes what relaybus opened itself at the deadline and resolves with a `StopResult { graceful, abandonedDeliveries, durationMs }`. NestJS `app.close()` is bounded the same way. | | `@relaybus/eslint-plugin` | New package. `no-external-call-in-transaction` and `publish-requires-transaction`, both `error` in the recommended config. | | `@relaybus/otel` | New package. OpenTelemetry metrics (publish, claim, handler, end-to-end latency, retries, dead letters, backlog gauges, a counter of undecodable inbound messages) and traces across services. [Observability](https://berkcansavur.github.io/relaybus/concepts/observability). | | Observer hooks | Timed hooks (`onPublishFinished`, `onClaimFinished`, `onHandlerStarted`, `onHandlerFinished`, `onTransportSent`, `onReceiveFinished`, `onInvalidMessage`, `onBacklog`), skipped entirely when no observer implements them. | | Testing | Every timer of the packages runs through the `Clock`, so `FakeClock` controls them all. | | Benchmarks | `bench:full` gains two routed cells (`distinct:1000` and `hot:10` keyed events sent to an in-process fake transport that acks after 5 ms). | ### Breaking changes | What | 0.2.0 | 0.3.0 | Action | | ------------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Route handler ids | `relaybus.transport:::` | `relaybus.transport::` | Update dashboards and code that match route ids (`handlers()`, `listDeliveries({ handler })`, `HandlerContext.handler`, observer events). Stored 0.2.0 deliveries are still sent. | | Routes to one destination | Different `retry` or `timeout` were allowed | They must be equal, and an event type may be routed to a destination once; otherwise `new RelayBus` throws `ConfigurationError` | Give such routes the same settings. | | Transaction time budget | No limit | `transactionTimeout`, default `'5s'`: a longer `bus.transaction` or `@RelayBusTransaction()` method is rolled back with `TransactionTimeoutError` | Move slow calls out of the transaction, or raise the option. | | `bus.stop()` | Returns `Promise`, waits without a deadline | Returns `Promise`, bounded by `timeout` (default `dispatcher.shutdownTimeout`, 30 s) | Only code that types the result as `void` needs a change. | | Error names | Five Kafka and RabbitMQ errors reported the name `RelayBusError` | They report their own class name | Code that matched `error.name === 'RelayBusError'` on dead letters must use the class name or `code`. | | Failure stage | `onFailure` stages without `'backlog'` | New stage `'backlog'` | An exhaustive `switch` over the stage needs a new case. | ### Deprecations Removed in 1.0.0; they still work and behave the same. - `@Subscribe` is the earlier name of `@RelayBusConsumer`; `@EventConsumer` of `@RelayBusCatalogConsumer`. Their option and metadata types are deprecated too. - The inbound-only `decode` option of `KafkaTransport` and `RabbitMqTransport`. Use `codec`; giving both is a configuration error. ### Benchmark: 0.2.0 vs 0.3.0 In short: 0.3.0 is as fast as 0.2.0 on publish and on MongoDB, and about 1.8 times faster for keyed PostgreSQL dispatch in two cells. No cell showed a lost, duplicated or out-of-order delivery. Most cells have one sample per side, so a difference below about 10 % is noise. The full tables and caveats are in [Performance: 0.3.0 vs 0.2.0](https://berkcansavur.github.io/relaybus/performance#_0-3-0-vs-0-2-0). Method: `pnpm bench:full --base v0.2.0`, interleaved on one machine (Apple M4, Docker Desktop, PostgreSQL 17, MongoDB 7, Node 24.3.0), fixed windows, database counters and exactness validation, as described in the [benchmark README](https://github.com/berkcansavur/relaybus/blob/main/benchmarks/README.md). The full run (39 minutes, all cells) used commit `a8587ff`; every sample was valid except the two of the PostgreSQL fan-out cell on four instances, whose preloaded backlog was too small on both sides, so that cell is left out. The publish and mixed-load cells were measured again after two publish fixes, with 2 samples per side and the 0.2.0 side taken from stored 0.2.0 samples. | Scenario | 0.2.0 per second | 0.3.0 per second | Ratio | Samples | | ------------------------------------------------- | ---------------: | ---------------: | -----------: | :-------------------: | | PostgreSQL publish, 1 publisher, 1 event per call | 1,543 | 1,573 | 1.02 | 2 / 2 | | PostgreSQL publish, 32 publishers, 1 event | 6,235 | 6,111 | 0.98 | 2 / 2 | | PostgreSQL publish, 8 publishers, 1,000 keys | 3,789 | 4,145 | 1.09 | 2 / 2 | | PostgreSQL mixed load, 5,000 per second, tuned | 4,133 | 4,032 | 0.98 | 2 / 2 | | PostgreSQL routed keyed dispatch, 1,000 keys | 3,267 | 5,827 | 1.78 | 1 / 1 | | PostgreSQL keyed dispatch, 1,000 keys, 4 workers | 4,453 | 7,931 | 1.78 | 1 / 1 | | PostgreSQL and MongoDB publish, all other cells | - | - | 0.94 to 1.09 | 2 / 2 | | PostgreSQL dispatch, all other cells | - | - | 0.88 to 1.19 | 1 / 1, one cell 3 / 3 | | MongoDB dispatch, all cells | - | - | 0.98 to 1.06 | 1 / 1, one cell 3 / 3 | Statements per publish are 4 in both versions (4.07 against 4.07 at one publisher), so the transaction budget costs no extra round trip. Mixed-load p99 ratios are not meaningful at 5,000 per second: both sides are saturated and p99 moved between 79 ms and 10.6 s from run to run. Two things to know. First, PostgreSQL WAL bytes per delivery were 8 to 11 % higher in four dispatch cells and 5 to 8 % lower in five others (one sample each, except one cell with three), so there is no clear direction yet. Second, the first full run showed 25 % more statements per publish (5 against 4). That was the transaction budget and was fixed before this table was taken, in `a13fb7e` and `2fb3cde`. The publish cells that were not measured again (8 publishers with batches, five handlers, hot keys) were 0.79 to 1.02 in the full run, before the fix. ### Known issues - **p99 latency near saturation.** Carried over from 0.2.0 and not changed by this release: at about 5,000 events per second on a laptop, end-to-end p99 on PostgreSQL was higher than 0.1.0's. Re-measure with `bench:full` for your hardware. - **Order across the upgrade.** Deliveries stored under 0.2.0 route ids are a separate queue from new ones. See [Upgrading](https://berkcansavur.github.io/relaybus/upgrading#order-during-the-upgrade). - **RabbitMQ and the shutdown deadline.** A connection opened with `url` is cut at the deadline, even during the TCP or AMQP handshake. A `connect` function you supply gets `connect({ signal })` and must honour it; a socket it keeps opening is not stopped by relaybus. Kafka has no forced disconnect, so past the deadline the bus stops waiting for it. - **`TimeoutNegativeWarning` on Node 23 and newer.** `kafkajs` can print this warning; it comes from kafkajs, not from relaybus. - **ORM adapters do not use prepared statements.** Unchanged from 0.2.0; run dispatchers on a `pg` pool ([Tuning](https://berkcansavur.github.io/relaybus/tuning)). ## 0.2.0 (2026-10-24) **Breaking.** Completed deliveries are now deleted when they complete instead of being kept. To keep them for `listDeliveries`, pass `completionHistory: true`. The 0.1.0 PostgreSQL and MongoDB schemas are upgraded in place by `setup()`. Follow [Upgrading from 0.1 to 0.2](https://berkcansavur.github.io/relaybus/upgrading); stop every 0.1.0 worker first. ### What is new | Area | Change | | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Delivery path | Workers lease 256 _lanes_ instead of rows, claim and complete in batches, and delete a delivery when it completes. The delivery row is written once and deleted once. [Why](https://berkcansavur.github.io/relaybus/performance/write-amplification) | | Throughput | On one laptop against 0.1.0: PostgreSQL publish 1.1 to 1.9x, PostgreSQL dispatch up to 18x, MongoDB dispatch 2 to 26x, about half the WAL per delivery. [Numbers and method](https://berkcansavur.github.io/relaybus/performance) | | Deadlock-free locking | One documented lock order for every statement, and permanent stress tests: 10 dispatchers on PostgreSQL and on MongoDB, concurrent publishers, takeovers and keyed ordering. | | Wake-ups | PostgreSQL `LISTEN`/`NOTIFY` (`wakeUp: 'notify'`) starts new work on other processes within milliseconds; at most one `NOTIFY` per 10 ms per store. `wakeUp: 'poll'` turns it off. | | Prepared statements | The statements relaybus repeats are parsed once per connection (`preparedStatements`, on by default for `pg` pools). [Why](https://berkcansavur.github.io/relaybus/performance/publish-path) | | Event catalogs | `defineCatalog`, `HandlersOf`, `defineHandlers`, `bus.forCatalog`, `bus.subscribeCatalog` and a boot-time coverage check. `consume(...)` and `produce(...)` roles let services share one catalog; `@EventConsumer` in `@relaybus/nestjs`. [Guide](https://berkcansavur.github.io/relaybus/concepts/event-catalogs) | | Foreign messages | Kafka and RabbitMQ `decode` option with `plainJsonDecoder` and `decodeCloudEvent` (structured mode). Default decoding is unchanged. [Guide](https://berkcansavur.github.io/relaybus/recipes/interoperating-with-other-services) | | MongoDB | Lane documents, bulk completion, smaller new documents. | | Tuning | New options `wakeUp`, `preparedStatements` and `completionHistory`. [Tuning guide](https://berkcansavur.github.io/relaybus/tuning) | | Tooling | Shared test containers (suite about 6 to 3.5 minutes), an interleaved benchmark harness (`pnpm bench:quick`, `pnpm bench:full`), performance budgets that can only go down. | Verification: 2,292 tests with 100 % statement, branch, function and line coverage; typecheck, lint, build and publint green; 0 lost, 0 duplicate and 0 out-of-order deliveries in the full benchmark. ### Known issues - **p99 latency near saturation.** At 5,000 events per second on a laptop, end-to-end p99 on PostgreSQL is higher than 0.1.0's (524 ms against 114 ms in the single-sample run), although 0.2.0 accepts more load there. Below saturation (1,000 per second) p99 is far lower. To be addressed in a later release. - **Ordering scope.** Order is guaranteed between deliveries of the same handler and the same ordering key. Unkeyed events, and deliveries to different handlers, have no relative order. - **`bus.stop()` has no deadline yet.** It waits for running handlers to finish; a handler that never returns holds shutdown until the process manager kills it. Give handlers their own timeouts (`timeout` on `@Subscribe`). - **ORM adapters do not use prepared statements.** Prisma, TypeORM, Drizzle, Knex and Kysely send every statement unprepared, so each publish and claim is parsed again (about 0.3 ms per publish and 1.3 ms per claim of 10 deliveries). Run dispatchers on a `pg` pool; see [Tuning](https://berkcansavur.github.io/relaybus/tuning). ## 0.1.x First public release: the transactional outbox, per-handler deliveries, retries, dead letters and ordering keys for PostgreSQL and MongoDB, the Kafka and RabbitMQ transports and the NestJS module. Upgrade with [Upgrading from 0.1 to 0.2](https://berkcansavur.github.io/relaybus/upgrading). ## How versioning works Every push to `main` with releasable [Conventional Commits](https://www.conventionalcommits.org/) publishes all seven `@relaybus/*` packages to npm with **one shared version**, computed by semantic-release. There is no committed changelog: the [GitHub Releases](https://github.com/berkcansavur/relaybus/releases) page is the changelog. - Current release line: **0.2.x**. While the version is below 1.0.0, a breaking change bumps the minor version. - `feat` commits bump the minor version; `fix`, `perf`, `refactor` and `revert` bump the patch version (a refactor changes the published code, never its behaviour). `docs`, `chore`, `ci`, `test`, `build` and `style` do not release. - Prereleases are published from the `next` and `beta` branches under the matching npm dist-tags. - Releases are published by CI with npm Trusted Publishing (OIDC) and provenance, so a tarball links back to the workflow run that built it. ## Install a specific version ```sh pnpm add @relaybus/core@0.3.0 @relaybus/nestjs@0.3.0 @relaybus/postgres@0.3.0 ``` Keep all `@relaybus/*` packages on the same version. ## Links - [Upgrading from 0.1 to 0.2](https://berkcansavur.github.io/relaybus/upgrading) - [Performance: 0.2.0 vs 0.1.0](https://berkcansavur.github.io/relaybus/performance) - [GitHub Releases](https://github.com/berkcansavur/relaybus/releases) - [npm: @relaybus/core](https://www.npmjs.com/package/@relaybus/core) - [Security policy](https://github.com/berkcansavur/relaybus/blob/main/SECURITY.md) - [Contributing](https://github.com/berkcansavur/relaybus/blob/main/CONTRIBUTING.md)