Connecting Laravel to Xero Without Losing the Tenant
Xero rotates its refresh token on every refresh, which turns two concurrent workers into a permanently disconnected organisation. That failure, and the four others that follow it.
The integration goes in on a Tuesday. It pushes invoices, it pulls payments, the finance team stops copying numbers between two screens and everybody is pleased. Eleven days later one customer's organisation is disconnected and cannot be reconnected without them going through the consent screen again. Nothing was deployed. Nobody touched the code.
What happened is that two of your queue workers refreshed the same token in the same second.
The refresh token is single use
Xero's access token lives for thirty minutes. That part everybody handles. The part that catches teams is that refreshing it gives you a new refresh token and immediately invalidates the one you sent. There is a sixty second grace period on the old one and after that it is gone.
So consider two jobs for the same tenant starting a few milliseconds apart, both finding an expired access token, both calling the refresh endpoint. The first gets tokens B and stores them. The second sent the same old token, and inside the grace window it also gets a valid response - a different pair, tokens C - and stores those over the top. Now the row holds C, Xero's most recent issue was C, and everything looks fine. Run it again under real load with the grace window already spent and the second call fails, your error handler writes nothing, and the row still holds a refresh token that was burned. The organisation is disconnected and the only fix is re-consent.
The defence is that exactly one process may refresh a given tenant at a time, and the others wait for its result rather than doing their own:
public function accessToken(XeroConnection $connection): string
{
if ($connection->expires_at->isAfter(now()->addMinutes(2))) {
return $connection->access_token;
}
return Cache::lock("xero:refresh:{$connection->tenant_id}", 30)
->block(20, function () use ($connection) {
$connection->refresh(); // re-read; someone may have won
if ($connection->expires_at->isAfter(now()->addMinutes(2))) {
return $connection->access_token;
}
return $this->exchange($connection);
});
}Three details in there are load bearing. The two minute margin means a token is
never handed to a job that will spend ninety seconds queued behind a slow
request. The re-read inside the lock is what makes the losing worker cheap - it
wakes up, sees a fresh token and uses it. And block rather than get means
the second worker waits instead of returning false and failing a job that had
nothing wrong with it.
Write the new pair in a transaction, and treat a failed exchange as a state change rather than an exception. If Xero says the refresh token is invalid, the connection is dead; marking it dead and telling the customer is the correct behaviour, and retrying it forty times is not.
A connection is not a company
The authorisation is not to a user and it is not to your customer's business. It
is to a tenant, identified by a tenantId that comes back from the connections
endpoint after consent, and one person clicking through the consent screen can
grant you three of them if they administer three organisations.
This matters on day one because it decides your schema. Every record you sync carries the tenant it belongs to, every API call sends that tenant's header, and every query that goes looking for "the Xero invoice for this order" filters on it. Teams that skip this because the first customer had one organisation end up writing a migration during an incident, which is the worst time to be adding a column to a table with two million rows.
It also matters because the set is not fixed. An organisation can be removed from your app inside Xero's own interface, by somebody who has never seen yours. Re-check the connections endpoint on a schedule, not only at consent, and reconcile it against your table.
The rate limit is four limits
You get sixty calls per minute per tenant, five thousand per day per tenant, ten
thousand per minute across your whole application, and a cap on concurrent
requests. They fail the same way, with a 429 and a Retry-After, but they mean
different things and the response tells you which one you hit in the
X-Rate-Limit-Problem header.
A per-minute limit is a pacing problem and the answer is to slow that tenant's queue down. A daily limit is a design problem and no amount of backoff fixes it
- you are fetching things you already have. The application limit is the one that ruins a Tuesday for every customer at once because one customer's initial import is running, which is the argument for a per-tenant queue rather than a shared one.
Respect Retry-After literally. Laravel's RateLimited middleware on the job,
released back with the exact number of seconds the header gave you, is the whole
implementation and it is better than any exponential backoff you would write,
because the number is not a guess.
Webhooks tell you that something changed, not what
Xero's webhook payload carries a resource type, a tenant, an id and a timestamp. It does not carry the invoice. You get the news and then you go and fetch it, which means a webhook is a hint to read, not a write.
Two things about the endpoint are unusual enough to be worth knowing before you build it. Xero validates it by sending an intent-to-receive payload that you have to answer correctly before the subscription turns on, and the signature check uses an HMAC over the raw body, so anything that re-serialises the request before you hash it will fail in a way that looks like a wrong key. And the delivery has a short patience: acknowledge with a 200 and nothing else, put the id on a queue, and do the fetching afterwards.
Then build the polling path anyway. A query filtered by If-Modified-Since
gives you everything that changed since your last successful sync, and it does
not care whether your endpoint was up. That one query is what makes an outage a
delay instead of a hole in your data.
Reconciliation is the project
The engineering above is a fortnight. The part that takes the rest is deciding what is true.
Your application has an invoice. Xero has an invoice. The finance team edited the one in Xero on Thursday because that is where they work. Someone raised a credit note against it. A payment came in for a different amount than either record expects because the customer paid three invoices in one transfer. None of that is an API question.
The decisions that have to be written down before code is useful: which system owns each field, what happens when both sides change the same one, whether a voided invoice in Xero voids the order in your application or only flags it, how a partial payment allocates, and what your invoice numbering does when Xero is the system of record for numbering in some jurisdictions and not others. Our integration engineering work is mostly this conversation, and the API calls are what falls out of it.
Make every write idempotent while you are here. Xero accepts an idempotency key, and the queue underneath you is at-least-once, so a duplicated job will otherwise become a duplicated invoice in somebody's accounts.
When not to build this
If the requirement is that the finance team sees revenue by product, an export and a spreadsheet delivers it this week and a two-way sync delivers it in two months. If the volume is forty invoices a month, the daily rate limit is not your problem and neither is any of the above - a nightly push is enough and it is a tenth of the code.
Build the synchronisation when the number of records makes manual entry a real cost, when both sides genuinely get edited, or when something downstream needs to react to a payment within minutes. Those are the cases where the reconciliation work pays for itself. Outside them it is a lot of engineering to remove a task that took someone twenty minutes a week.
If your ledger lives in Sage rather than Xero, most of this still applies and the access story does not - that is a different article, because Sage is not one product.
