Skip to content

Your Laravel Jobs Will Run Twice. Design For It.

At-least-once delivery means every queued job is eventually executed more than once. What that breaks, why retries are not the only cause, and how to make a job safe without a check-then-act race.

6 min read

Laravel's queue gives you at-least-once delivery. Almost every team reads that as "it retries on failure" and moves on, which is half of it. The other half is that a job which succeeded can still run again, and no amount of careful error handling prevents it.

Four ways a job runs twice

The retry you configured. A job throws, the queue retries it. Obvious, and the one everybody accounts for.

The timeout that was wrong. The job takes longer than the configured timeout. The worker is killed, the message is not acknowledged, another worker picks it up - while the first one may still be finishing its work.

The deploy. A worker is holding a job when the code changes under it. It is restarted; the message goes back on the queue. The job did most of its work before the restart, and all of it will be done again.

The infrastructure. A connection drops between "the job finished" and "the acknowledgement reached the broker". The work happened. The queue does not know that.

Only the first is under your control. The other three are why a queue is at-least-once and not exactly-once, and why exactly-once delivery is not something you can configure.

What this actually costs

An email sent twice is an embarrassment. Everything below is worse:

  • A payment captured twice, which is a refund, a support thread and a customer who does not come back.
  • A webhook delivered twice to a partner whose own system is not idempotent either.
  • A stock level decremented twice, which is an oversold product.
  • An invoice number allocated twice, which is an accounting problem with a regulator attached in some jurisdictions.

The last one stops being hypothetical the moment a queue is pushing records into someone else's ledger. A duplicated job against an accounting API is a duplicated invoice in a real set of accounts, and the person who has to unpick it does not work for you.

The pattern is the same in all of them: the effect happened outside your database, so a transaction could not undo it.

The fix that does not work

This is the code we find most often, and it is a race:

public function handle(): void
{
    if ($this->order->refresh()->paid_at !== null) {
        return;                       // already done, skip
    }
 
    $charge = $this->gateway->charge($this->order);   // ← both workers get here
 
    $this->order->update(['paid_at' => now()]);
}

Two workers read the row before either writes it. Both see null, both charge. Check-then-act is not a guard, it is a narrower window - and queue duplicates arrive in exactly the conditions that make windows narrow.

Guards that hold

The rule is that the uniqueness has to be enforced by something that cannot be raced: the database, or the third party.

A unique constraint doing the work. Let the insert fail rather than asking first:

public function handle(): void
{
    try {
        $payment = Payment::create([
            'order_id'        => $this->order->id,
            'idempotency_key' => $this->key,   // unique index
        ]);
    } catch (UniqueConstraintViolationException) {
        return;            // another worker owns this one
    }
 
    $this->gateway->charge($this->order, $this->key);
}

An idempotency key the provider enforces. Every serious payment API accepts one. Two identical requests with the same key produce one charge and two identical responses. This is the strongest guarantee available, because it holds even if your own database is the thing that failed.

A conditional update. Make the state transition itself the lock:

$claimed = Order::where('id', $this->order->id)
    ->whereNull('paid_at')
    ->update(['paid_at' => now()]);   // returns rows affected
 
if ($claimed === 0) {
    return;      // somebody else transitioned it
}

One statement, decided by the database. The worker that gets 1 owns the work.

A key generated at dispatch, not in the job. This matters and is easy to miss: if the job computes its idempotency key from the current time or a random value, two executions of the same job produce two keys and the guard never fires. The key is part of the job's payload, created once, when the job is dispatched.

While you are in there

Two settings and one habit, all cheap:

Set the timeout below the retry delay. If a job can run for 90 seconds and is retried after 60, you have guaranteed yourself concurrent executions of the same job.

Use backoff, not a fixed delay. public $backoff = [10, 60, 300]; - a third-party outage is not improved by hitting it three times in thirty seconds.

Do not retry what cannot succeed. A validation failure retried five times is five identical failures and a delay to everything behind it in the queue. Throw something the job treats as fatal, and let it fail on the first attempt.

The last mile: somebody has to be watching

Every fix above is pointless if failures are invisible. The single most common production fault we are called in to look at is a failed_jobs table with thousands of rows that nobody has ever read.

That is not a queue fault. The job failed, the framework recorded it exactly as documented, and nothing was attached to the recording. Wire failures to whatever your team already watches, alert on the rate rather than the event, and put a heartbeat on the queue itself so that a worker which stops is distinguishable from a queue that is simply empty.

A queue you can leave alone is not one that never fails. It is one that tells you when it does, and does no harm when it repeats itself.

If yours is currently the other kind, that is the work.

Related questions

Does setting tries to 1 make this go away?
No. It removes the retry, not the duplicate. A worker killed after the job did its work but before it acknowledged the message will see that message again, whatever tries is set to - and now you also have no retry for the transient failures that deserved one.
Is a database transaction enough?
It makes your own writes atomic, which is necessary and not sufficient. A transaction cannot roll back an email you sent or a charge you made at a payment provider, and those are exactly the effects a duplicate hurts most.
What about ShouldBeUnique?
It prevents a second job being dispatched while the first is pending, which stops duplicate dispatches. It does not stop a single dispatched job being executed twice, because the lock is released when the job starts running. It is useful and it is a different problem.
How do we know if this is already happening to us?
Look for the symptoms rather than the logs: duplicate emails reported by customers, webhook deliveries your partner received twice, rows that violate a uniqueness rule you thought was enforced. If your failed_jobs table has entries nobody has read, assume it is happening.

← Back to all articles

Call us+1 848 272 7583WhatsApp+90 850 308 5436Emailinfo@codefacture.comContact page