---
title: "Batch Plugin"
description: "Combine multiple requests into a single batch and receive their responses together, reducing the overhead of separate requests."
sidebar:
  label: "Batch"
---

:::warning
HTTP/2, HTTP/3, and later versions already support multiplexing, which allows multiple requests and responses to share a single connection. Because these protocols are now widely adopted, this plugin is often less useful than it once was.
:::

## Setup

Set up batching on both the server and the client. The server plugin handles incoming batch requests, and the client plugin groups outgoing requests into batches.

```ts server.ts
import { BatchHandlerPlugin } from '@orpc/server/plugins'

const handler = new RPCHandler(router, {
  plugins: [
    new BatchHandlerPlugin(),
  ],
})
```

```ts client.ts
import { BatchLinkPlugin } from '@orpc/client/plugins'

const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})
```

:::warning
`BatchHandlerPlugin` recognizes batch requests by the `orpc-batch` header (exported as `BATCH_HEADER`), so make sure nothing between the client and server blocks or strips it, such as a CORS policy or proxy. This is usually fine by default, as the [CORS Plugin](/docs/plugins/cors) allows the headers a request asks for.
:::

## Max Size

Both plugins limit how many requests a batch holds with `maxSize`, and each side defaults to `DEFAULT_BATCH_MAX_SIZE` (10) independently. The client splits a larger group into several batches, while the server rejects a larger batch with a `413` for every request in it, so keep the client limit at or below the server's.

```ts server.ts
const handler = new RPCHandler(router, {
  plugins: [
    new BatchHandlerPlugin({
      maxSize: 20,
    }),
  ],
})
```

```ts client.ts
const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      maxSize: 20,
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})
```

## Response Modes

In the default `buffered` mode, the server starts the batch response once every procedure has responded, so it can carry their [headers](#response-headers). The response streams only if one of them returned an [AsyncIteratorObject](/docs/async-iterator-object) or a [`ReadableStream`](/docs/binary-data). If your client cannot read streamed responses, keep these procedures out of the batch with [`filter`](#filtering-requests).

In `streaming` mode, the server starts the response once the first procedure responds and sends each other response as soon as it is ready, so a slow procedure does not hold back the rest:

```ts
const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      mode: 'streaming',
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})
```

:::warning
A `streaming` batch response always streams, so your client and any proxy in between must support streamed responses. It also carries only the cookies of the first procedure to respond. Send procedures that set cookies, such as a login, outside the batch with [`filter`](#filtering-requests).
:::

### Keep-Alive Timer

A streaming batch response may stay idle for long periods. The server plugin can send keep-alive frames to keep the connection alive.

```ts
const handler = new RPCHandler(router, {
  plugins: [
    new BatchHandlerPlugin({
      keepAlive: {
        /**
         * If true, a keep-alive frame is sent periodically while the stream is idle.
         *
         * @default true
         */
        enabled: true,
        /**
         * Interval (in milliseconds) between keep-alive frames after the last message.
         *
         * @default 15000
         */
        interval: 15000,
      },
    }),
  ],
})
```

## Compression

A batch response might be left uncompressed by the [Response Compression Plugin](/docs/plugins/response-compression), depending on the shape it takes. On Node.js, add the [Batch Response Compression Plugin](/docs/plugins/batch-response-compression) when you know your batch responses are all compressible. It covers every one of them, without giving up streaming:

```ts
import { BatchResponseCompressionHandlerPlugin } from '@orpc/node'

const handler = new RPCHandler(router, {
  plugins: [
    new BatchHandlerPlugin(),
    new BatchResponseCompressionHandlerPlugin(),
  ],
})
```

## Groups

Only requests in the same group are batched together. Each group also defines a context, as described in [client context](/docs/rpc/link#client-context).

The following example batches requests by cache policy:

```ts
interface ClientContext {
  cache?: RequestCache
}

const link = new RPCLink<ClientContext>({
  method: ({ context }) => {
    if (context?.cache) {
      return 'GET'
    }

    return 'POST'
  },
  plugins: [
    new BatchLinkPlugin({
      groups: [
        {
          condition: ({ context }) => context?.cache === 'force-cache',
          context: { // used for the rest of the request lifecycle
            cache: 'force-cache',
          },
        },
        { // Fallback for all other requests. Keep this last.
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
  fetch: (url, init, { context }) => globalThis.fetch(url, {
    ...init,
    cache: context?.cache,
  }),
})
```

Now, calls made with `cache = 'force-cache'` use that cache setting whether they are batched or sent individually.

## Filtering Requests

Use `filter` to skip batching for specific requests before group matching runs. Requests for which `filter` returns `false` continue through the link chain individually.

```ts
const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      filter: ({ path }) => !path.includes('upload'),
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})
```

## Wait Time

By default, only requests made in the same [event loop](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Execution_model#job_queue_and_event_loop) tick are batched together. Set `wait` to keep collecting requests for a number of milliseconds, trading a little latency for fewer, larger batches.

```ts
const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      wait: 10, // ms
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})
```

The wait starts when the first request is queued, so no request waits longer than `wait`, even while more keep arriving.

## Scope

On the server, requests from different users can be batched together. If your transport reads per-request data (in `fetch`, `origin`, and so on), set `scope` to a value unique to the incoming request.

```ts
import { AsyncLocalStorage } from 'node:async_hooks'

const requestStorage = new AsyncLocalStorage<Request>() // set per incoming request

const link = new RPCLink({
  url: '/rpc',
  plugins: [
    new BatchLinkPlugin({
      scope: () => requestStorage.getStore(),
      groups: [
        {
          condition: () => true,
          context: {},
        },
      ],
    }),
  ],
})
```

:::tip
Per-request data passed through the link's [`headers` option](/docs/rpc/link#url-and-header-options) is resolved for each request, so it is safe without `scope`.
:::

## Response Headers

In `buffered` mode, the batch response carries the cookies and `Vary` fields of its subresponses, and the headers they all share. In `streaming` mode, it carries only the cookies of the first subresponse to respond. Other headers stay on their subresponse. To change what stays on each subresponse, set the server plugin's `mapSubresponse` option.

## Learn More

See the [BatchHandlerPlugin source code](https://github.com/middleapi/orpc/blob/main/packages/server/src/plugins/batch.ts) and the [BatchLinkPlugin source code](https://github.com/middleapi/orpc/blob/main/packages/client/src/plugins/batch.ts) for implementation details.