Skip to main content

Concurrency policies

Two requests can hit the same session at once. A webhook fires twice a second apart. A user sends a second message before the first reply lands. Two browser tabs share one conversation. Without a rule, both requests resolve the session and run in parallel, racing to write the same state.

A concurrency policy decides what happens in that moment. You declare it on an action, or as a flow-wide default, and the framework arbitrates competing requests on a key you choose. By default the key is the session.

It sits next to idempotency. Idempotency stops the same delivery from running twice. A concurrency policy arbitrates two different requests that collide on one key. Webhooks usually want both, and they compose.

The policies​

  • allow runs requests concurrently. It's the default. Reach for it on cheap, append-only work where ordering doesn't matter.
  • queue runs requests on the key one at a time, in arrival order (first in, first out). One finishes before the next starts. Use it for chat: a quick burst of messages gets ordered, coherent replies instead of racing duplicates. A run that waits more than 30 seconds for its turn fails with ConcurrencyQueueTimeoutError (status 503), which is safe to retry with backoff.
  • reject drops a competing request while another one holds the key. Use it for webhook double-fire: the duplicate is dropped, not queued. How the dropped caller hears about it depends on the transport, covered below.
  • hold runs at once, like allow, and marks the key busy while it runs. It never waits, whatever else holds the key, and is never refused. Put it on a turn that follow-up work must not overlap, such as a reply in a conversation.
  • defer waits until nothing on the key is running or waiting, then runs. Put it on follow-up work that should land after the current turn, not beside it. See Follow-up work after a reply.

debounce (collapse a burst into one run) and restart (cancel the in-flight run) are reserved names, not implemented yet. Declaring either throws at definition time. See Coming next.

What allow leaves to you​

Parallel runs are safe from data corruption. An update computed from current state is refused and retried if another run moved the record first, and an increment or an append is applied to whatever the store holds. What allow doesn't prevent is one run overwriting another's value on the same field, or two replies to one session interleaving. If ordering matters, use queue or reject.

A control sent beside a running request​

Sometimes an allow action exists to talk to a request that's already running, such as a line handed into a turn in progress. By default every request becomes the newest on its session, and that's the one a client reattaching to the session (useSession with autoResume) looks for. A short control request would take that place, so a page reloaded mid-turn would find the control, already finished, and not reconnect to the turn. Set autoResume: false on the control's action and the session keeps pointing at the request before it:

actions: {
turn: { block: turn, concurrency: "queue" },
steer: { block: steerRunningTurn, concurrency: "allow", autoResume: false },
},

The control's own request is still recorded and listed with the session's requests. Only the reattach target stays where it was.

Setting a policy​

Set concurrency on an action, or request.concurrency for a flow-wide default every action inherits:

defineFlow({
kind: "support-chat",
request: { concurrency: "queue" }, // flow-wide default
actions: {
appendMessage: { block: appendPipeline, concurrency: "allow" }, // cheap, always runs
respond: { block: respondPipeline }, // inherits "queue"
syncInvoice: { block: invoicePipeline,
concurrency: { policy: "reject", key: "user" } },
},
});

The action's own setting wins: action.concurrency ?? flow.request.concurrency ?? "allow". Leave both unset and every action runs as allow.

The same declaration governs every way a request arrives: HTTP, webhooks, schedules, MCP and dispatches from other flows. You don't wire it per transport.

Keying​

A policy arbitrates requests that share a key. By default that's the session, so two requests on one conversation contend and requests on different sessions never do. Override it with key:

  • "session" (default): the session id, namespaced by tenant. Resolves to no key when the request has no session.
  • "user": the user id, namespaced by tenant. One in-flight run per user across all their sessions.
  • "none": no arbitration for this action.
  • a function: (ctx) => string | undefined. Derive a custom key, such as a webhook delivery id pulled from metadata. Return undefined to opt the request out.
// One sync per user at a time, dropping duplicates.
syncInvoice: { block: invoicePipeline, concurrency: { policy: "reject", key: "user" } },

// Dedup webhook deliveries by the provider's delivery id. The webhook
// transport namespaces it under `metadata.webhook.deliveryId`.
onEvent: { block: handlePipeline, concurrency: {
policy: "reject",
key: (ctx) => (ctx.metadata?.webhook as { deliveryId?: string } | undefined)?.deliveryId,
} },

When the key resolves to undefined, the request runs as allow. That happens with no session under the "session" key, with "none", and when a key function returns undefined. MCP calls carry no session, so they run unarbitrated under the default key.

Chat: split append from respond​

Appending an inbound message to a session is cheap and should always succeed. Generating a response is the expensive, contended part. Split them into two actions and each gets the policy it needs:

defineFlow({
kind: "support-chat",
actions: {
appendMessage: { block: appendPipeline, concurrency: "allow" }, // every message lands
respond: { block: respondPipeline, concurrency: "queue" }, // replies stay ordered
},
});

A burst of messages all append immediately, and the queued respond answers them in order over the accumulated history. No racing replies, no state-collision errors. Collapsing the whole burst into a single reply is what debounce will add.

Follow-up work after a reply: hold and defer​

Some work arrives for a conversation from somewhere other than the person in it. A background job finishes, and the app wants the assistant to say so. If that follow-up starts while the assistant is still replying, the conversation shows two replies streaming at once. queue would stop that, but it would also make the person's own next message wait behind the reply, and fail after 30 seconds.

hold and defer give the two kinds of work different rules. The person's turn runs at once and holds the session. The follow-up defers to it:

defineFlow({
kind: "assistant",
actions: {
respond: { block: respondPipeline, concurrency: "hold" }, // a person's message
reportResult: { block: reportPipeline, concurrency: "defer" }, // a finished job's follow-up
},
});

What happens on one session:

While this runsA hold request (respond)A defer request (reportResult)
nothingruns at onceruns at once
a hold requestruns at once, alongside itwaits until every running hold has ended
a defer requestruns at once, alongside itwaits its turn: deferred requests run one at a time

A deferred request waits for every hold that is running, including one that started after it began waiting. Its request id and stream are available at once, and the stream stays open while it waits. If the caller cancels it while it waits, it never runs.

The wait has no time limit. It ends when the request ahead of it ends, however that happens: it finishes, it throws, or it's cancelled. If a server crashes while it runs the request ahead, a deferred request waiting on another server starts once the crashed server's lease on the key runs out (ten seconds by default with bullmqWorker).

queue and reject requests on the same key see a running hold too: a queue request waits for it, and a reject request is refused while it runs.

Two bounds keep a busy session from piling up deferred work:

  • At most 32 defer requests wait on one key in each server process. On a queue, a request counts from when it's accepted until its job ends, so one that's running still counts. The next one is refused the way reject refuses: 409 over HTTP, a skipped 200 for webhooks and schedules. Nothing is created for it, so retry it once the session quiets down.
  • A defer request waits for newer replies for 30 seconds at most. After that it waits only for the replies already running, then runs, even if the person keeps sending messages.

What defer won't do:

  • Once a deferred request starts, it doesn't wait for anything else. A message sent while it runs starts at once, beside it.

Webhooks: dropping the double-fire​

Providers retry, and retries can arrive while the first delivery is still running. A reject policy keyed on the session, or on the delivery id, drops the duplicate:

defineFlow({
kind: "billing",
request: { concurrency: "reject" },
actions: {}, // no caller-facing actions: webhooks are the only way in
webhooks: { /* ... */ },
});

What the dropped caller sees depends on how it arrived:

TransportA rejected request gets
Webhook200 { status: "skipped" }, so the provider stops retrying instead of counting a failure
Schedulea skipped 200
HTTP409 carrying the in-flight requestId, so a client can tail the run that won instead of retrying blindly
MCPa server-busy error

reject only catches a duplicate that arrives while the first is running. A redelivery minutes later, after the first finished, needs an idempotency key.

Requests that pause and resume​

A request that suspends to wait for a person gives up its place on the key, so other requests on the session can run while it waits. When it picks up again, whether from the person's answer, a stop, or the sweeper, it takes a new place on the key like a fresh request.

  • Under queue, reject and defer it waits for its turn, then runs. Under reject it isn't dropped. The wait has the same 30-second limit as queue. If the session is still busy after that, the request stays paused with the answer saved, and the sweeper continues it on its next pass, every ten minutes by default.
  • Under hold it runs at once and marks the key busy, as any hold request does.
  • Under allow it runs at once.

The call that sent the answer doesn't wait for the turn. It gets its usual reply straight away: 202 with { requestId }, or an event stream if it asked for one. The answer is accepted at that point, so a resumed request that runs out of time doesn't fail with the 503 a new queue request gets. Its stream closes with no events, and the request stays suspended until the sweeper continues it.

While a resumed request waits for its turn, session.abortRequest() doesn't stop it. The request no longer waits on its gate, so the stop can't resolve that, and it isn't running yet: the call fails with a 409 that says the request was resumed before the stop reached it. The request's status reads suspended until it starts and in_progress once it runs. Call abortRequest() again then.

Across processes​

Where the policy holds depends on how your server runs requests.

  • One server, no queue. The policy is enforced in memory, in the process that runs the request.
  • A queue-backed deployment, such as bullmqWorker. The policy holds across every process: the web process and each worker. Details below.
  • Several web servers, each running requests in process, with no queue between them. Each server keeps its own keys, so two requests on one session can run at once if they land on different servers. Run that shape as a single instance, or put a queue in front of it.
  • A runner you pass to createFlowState directly, rather than through a worker adapter. It applies no policy to the work it dispatches, and a delivery into an existing session is refused with external-dispatcher.

On a queue​

A run takes its place on the key when it's accepted, then waits for its turn in whichever worker picks it up. Runs start in the order they were accepted. A worker never spends one of its slots waiting: a run whose turn hasn't come goes back on the queue for at most two seconds, then is checked again. The 30-second wait limit is the same as on one server, and it counts only time spent waiting for the key, not time queued behind unrelated work.

hold and defer work the same way on a queue as on one server. A hold request runs as soon as a worker takes it, and the key reads busy from the moment it is accepted until it ends. A waiting defer request never ties up a worker, the same as a waiting queue run. For its first 30 seconds of waiting it doesn't hold the key, so queue requests don't wait behind it and reject requests aren't refused because of it. Once it lines up behind the running replies, or starts, it holds the key like any other run.

A place on the key has a lease, which the worker running the job keeps renewing.

  • If that worker dies, the key frees once the lease runs out (ten seconds by default), and the next run starts.
  • If the worker is alive but can't reach Redis to renew, it aborts the run's signal at half the lease, and the run ends interrupted. A run that honours ctx.signal has stopped before anyone else can take the key. One that ignores it can overlap the next run.
  • If Redis can't be reached when a run needs a place, the run isn't started and the caller is told why. A run never goes ahead without the policy.

Lease length, rollout order and sizing are covered in the @flow-state-dev/bullmq README.

How it relates to other primitives​

  • Scheduled onOverlap is the scheduled-action spelling of the same idea. onOverlap: "skip" is reject keyed on the schedule id, and onOverlap: "allow" is allow. See Scheduled actions.
  • The state layer decides how two concurrent writes to one record resolve. A write built on a stale read is refused and re-applied against the value that won. An increment or an append is handed to the store as the operation itself. A plain single-field write is last-write-wins. See State Operations for which calls get which. A concurrency policy sits above all of that: the state layer resolves a write that already happened, and the policy decides whether the second run starts at all.

Coming next​

  • debounce will collapse a burst of arrivals into a single run. It never cancels a running run. Until it ships, use queue for chat bursts.
  • restart will cancel the in-flight run and start fresh on the newest request. The runtime doesn't roll back state a cancelled run already committed, so cancelling mid-write could leave a record half-updated. It stays unavailable until that can be handled safely.