Rolling back a move that went wrong

Roll back when you cannot yet name the fault. Fix forward when you can. Severity feels like the deciding variable at 2am, and it is not: a site throwing a loud, obvious error with a known cause is often twenty minutes from repair, while a site that is only slightly wrong for reasons nobody can explain can stay wrong for two days. Ask "do I know what this is", not "how bad does it look".

Which way each situation points

What keeps the option open

Rollback is not a technique you perform on the night. It is a set of things you refused to throw away beforehand, and each one is easy to lose by tidying up too early:

• The old environment left running and serving, not merely backed up. An archive is not a rollback target; it is a rebuild project.
• Every value you are about to change, recorded at its old setting, alongside the shortened record lifetime and the time you shortened it.
• A database export taken at the exact moment writes froze, with a readable timestamp in the filename.
• Old-host access and billing valid well past the window. Thirty days after cutover, not three.
• A certificate on the old host that has not expired, or going back lands visitors on a browser warning instead of a working site.

Most of this belongs in the pre-cutover assembly covered under launch playbooks, because it costs nothing then and cannot be added later.

The part that actually hurts

Anything written to the new environment after cutover disappears when you go back. On a brochure site that is nothing. On a store taking forty orders a day, six hours of trading is roughly ten orders that exist in one place only. You have three honest options: freeze writes for the whole observation window, export the delta and replay it after the rollback, or accept the loss and reconstruct from confirmation mail.

Replay is the one people overestimate. It works only when every new row carries a timestamp you can filter on, and it breaks when auto-increment identifiers have moved on both sides, because the old database will happily reissue numbers the new one already gave to somebody. If the schema itself was migrated, replay is usually off the table.

Decide the deadline before you cut over

Write the abort time down while everyone is calm, and give one named person authority to call it. Rollback consumes time too, so it lives inside the window rather than after it: with a four-hour window and a rollback that takes forty-five minutes, the decision moment is three hours and fifteen minutes in, not four. Miss that and you have not chosen to fix forward, you have simply run out of the alternative.

When going back stops being possible

The door closes gradually, then completely. It is shut once the old environment is gone or unpaid, once a registrar move has completed, once a schema change has been followed by real writes, or once the new data is worth more than the outage. Past that line the work changes character: one change at a time, each written down, no parallel edits by two people, and no destructive commands until the cause is understood. That is diagnostic work, and rushing it is how a bad night becomes a bad week.