Laravel development for San Francisco product teams

San Francisco is the widest time gap we work across, and it removes the thing most teams rely on for risky work: somebody watching.

At ten hours of difference there is simply no hour at which both teams are at their desks. A migration that starts during our afternoon finishes while San Francisco is asleep, and one timed for your morning runs with nobody here. That is usually described as a disadvantage, and for a backend engagement it mostly forces a discipline that was a good idea anyway.

A San Francisco schema change that nobody is watching

If a migration cannot be supervised, it has to be built so supervision is unnecessary. That means abandoning the single-step change entirely.

The pattern is expand and contract, and each step leaves a working system. Add the new column, nullable, which changes nothing. Backfill it in batches that check replication lag and can stop and resume. Deploy code that writes to both the old and new columns. Deploy code that reads from the new one. Leave the old column in place for a week. Remove it in a separate change once nothing has referenced it.

Six deployments instead of one, every one of them reversible, none of them requiring anybody to be awake. That is slower on paper and faster in practice, because the alternative - a single change that takes a lock on a large table - is the thing that produces the incident everybody remembers.

The migration that took the site down is the longer version, and over this distance it stops being advice and becomes the only option.

The backfill is the part San Francisco teams get wrong

Adding a column is instant on a modern setup. Populating three hundred million rows is not, and it is where unattended work goes wrong.

A backfill written as one statement will hold a transaction open, inflate replication lag, and either time out or take the site with it. Written as chunks, with a recorded cursor, a check on replica lag between batches and the ability to stop cleanly, it becomes a job that can run for two days in the background and be paused when something more important needs the database.

Resumability is the requirement that matters most here. A backfill that has to start over if it is interrupted is a commitment nobody can make across a ten-hour gap.

One bounded Laravel problem, written down

San Francisco buyers have their own senior engineers, which settles what kind of engagement makes sense.

Broad augmentation does not survive this time difference. Questions queue overnight, context is rebuilt every morning, and two capable teams spend their energy coordinating. What works is a single named problem with a boundary and a definition of done - a queue that turns out not to be idempotent, a version upgrade that has been deferred three times, a table that has become too large to change casually.

With no call available as a fallback, the written plan is the first deliverable rather than a formality. Your engineers should be able to reject it on its own terms while we are asleep, which is a higher bar than explaining it would have been.

A San Francisco deploy has to be reversible without a call

The release goes out while the client's office is dark. If it is wrong, nobody is going to wake up and coordinate a rollback with us, so the deploy has to be reversible by a single action taken by whoever notices.

That rules out a class of migration. A column renamed in one step breaks the previous application version the moment it lands, so rolling back the code leaves it pointing at a schema that no longer matches. The discipline is to expand first and contract later - add the new column, write to both, read from the old, deploy; then switch reads, deploy; then stop writing the old one and drop it, days later, once nothing has gone wrong. Three dull deploys, each individually reversible, instead of one that is not.

Behaviour changes go behind flags for the same reason. A flag can be turned off by somebody who does not know how the deployment works, at the moment they notice a problem, without a release. That is worth the small amount of conditional code it costs, and it is worth deleting the flag a fortnight later rather than accumulating a hundred of them.

The last piece is knowing whether the deploy was fine, which cannot depend on somebody looking. Error rate and queue depth before and after, compared automatically, with a threshold that notifies rather than a dashboard somebody has to open.

Laravel queues when the San Francisco team is asleep

A job fails at two in the morning Pacific. Nobody is going to look at it for seven hours, so whatever happens next has to happen without a person.

Retries with backoff handle the transient cases, which are most of them, but only if the job is safe to retry - which means the write it performs is idempotent rather than merely unlikely to be repeated. A job that charges a card and then records the charge will, on retry after a crash between the two, charge twice. The sequence has to be arranged so that the repeated attempt collides with something rather than succeeding.

What fails permanently needs to fail loudly and completely. A half-applied job is worse than a failed one, so work that spans several writes either runs inside a transaction or is broken into steps that each leave the record in a state the next step can resume from. failed_jobs is useful only if the row carries enough to diagnose without reproducing - the payload, the exception, and the identifiers of what it was working on.

Alerting is set on the shape of the queue rather than on individual failures. One failed job overnight is noise; a hundred, or a queue whose oldest unprocessed item is four hours old, is a thing somebody needs to know about before the working day starts in California. Those two thresholds, wired to something that actually reaches a person, are the difference between a quiet night and a morning spent reconstructing what did not happen.

Commercially

Scope fixed in writing before work starts, priced per project. The country-level ground - state privacy laws, sales tax, invoicing - is on the United States market page.

Invoiced by Codefacture Yazılım A.Ş. in Türkiye, in US dollars by bank transfer, with no Turkish VAT under the export-of-services exemption and no US entity on our side.