A Queue consumer gets batches; ack per message, retry only failures

send() enqueues bodies up to 128 KB; queue(batch) receives up to max_batch_size messages. Without per-message ack(), one failure retries the whole batch.

Queues & Crons

· Chapter

46

·

3

min read

The answer. Queues decouple the request from the work. A producer binding's send(body, { delaySeconds }) accepts any structured-cloneable value up to 128 KB; the consumer Worker exports queue(batch, env, ctx) and receives batch.messages, each with id, timestamp, attempts (starting at 1) and body. Delivery is at least once, so the consumer must be idempotent (a ledger row keyed by message id or by your own job id). The rule that costs people the most: if the handler throws or returns without acking, every message in the batch that was not explicitly ack()ed is redelivered, including the ones you already processed. Call msg.ack() after each success and msg.retry({ delaySeconds }) after each failure; after max_retries a message goes to the dead_letter_queue or is dropped if none is set.

The pattern.

[[queues.consumers]]
queue = "image-jobs"
max_batch_size = 10          # 1 for expensive LLM jobs, so a retry never re-runs nine successes
max_batch_timeout = 5        # seconds to wait for a full batch
max_retries = 5
dead_letter_queue = "image-jobs-dlq"   # created on deploy if missing
retry_delay = 30             # default backoff; per-message retry() can override
export default {
  async queue(batch: MessageBatch<Job>, env: Env) {
    for (const msg of batch.messages) {
      try { await process(msg.body, env); msg.ack(); }
      catch (e) { console.error(JSON.stringify({ id: msg.id, attempt: msg.attempts, error: String(e) }));
                  msg.retry({ delaySeconds: 60 * msg.attempts }); }
    }
  },
} satisfies ExportedHandler<Env, Job>;

Watch out.

  • One consumer Worker per queue; a second one fails to deploy. Fan out inside the consumer or use several queues.
  • Repeated failures push consumer concurrency down toward one. When a rate-limited downstream is the real constraint, cap max_concurrency yourself.
  • A DLQ nobody consumes is a delay, not a fix: messages there expire too. Alert on DLQ depth.

Related: queue-work-returns-202-client-polls-a-status-row · waituntil-buys-thirty-seconds-not-forever · cron-one-tick-many-isolated-legs