Cutover without downtime: migrating users while the old VPN stays up
The new mesh works. Now move real people onto it without a bad day. This part is the discipline that makes a migration boring: a real pilot, simpler onboarding, a wave-by-wave cutover with the old VPN as a live safety net, and backups you've actually restored.
Real client work, anonymized. Generic domains and hosts throughout.
By Part 4 the new stack was complete: SSO with a second factor, a WireGuard mesh, internal routes and DNS, and access scoped by team. But “the stack works” and “the users are migrated” are very different milestones, and the gap between them is where careful projects still cause outages. This part is about making the cutover boring, which is about the highest compliment you can pay a migration.
The mental model: additive, reversible, and proven before it’s trusted¶
One principle has run through this whole series, and it comes due now.
Why the old VPN stays up through the entire cutover
Every step so far has been additive. The new stack went up beside OpenVPN, never on top of it. So cutover isn’t a switch you flip. It’s people walking across a bridge that’s already been load-tested, while the old bridge stays open behind them. Nobody stops using OpenVPN until they’re confirmed working on the mesh. If a wave goes sideways, those users are still on the old VPN, so there’s nothing to “roll back,” because nothing was torn down. The new thing has to earn each user’s trust before it gets it.
First, a real pilot, not a demo¶
Pick two or three friendly users, and deliberately include one on a high-latency link (here, an office about 250 ms away) so you’re testing the transport under the worst conditions rather than the best. Each pilot installs the app, logs in with directory credentials and a second factor, confirms internal access and DNS, and confirms their OpenVPN still works alongside it.
Every pilot ran that path cleanly, and their OpenVPN kept working in parallel the whole time, so there was never a moment they were stranded.
Backups online before anyone depends on this
The moment real users rely on the new stack, it’s production, so backups come first rather than later. VM snapshots, a Postgres dump of Keycloak, and the mesh control-plane data directory, all shipped off the host and on a schedule. And do one test restore before the pilot ends. A backup you’ve never restored is a hope, not a backup.
Simplify onboarding while you’re in there¶
The old onboarding script SSHed to the VPN box, minted a client cert, and emailed a .ovpn file.
A migration is the natural moment to delete that whole ritual.
Why fix onboarding now instead of later
A new hire onboarded the old way gets a .ovpn you’ll only have to migrate again. So cut the
OpenVPN block out of the script now, keep the directory user, group, and home-directory
creation, and drop the cert dance. New users then land on the mesh by default: create the
directory account, Keycloak federates it automatically, and the user self-enrolls from a
one-page guide. That’s one fewer artifact and one fewer manual step, and it sets up the next
phase, which matches directory identities to the cloud IdP by email.
The end-to-end check passed with no .ovpn anywhere in it. Create the directory account, let
Keycloak federate it automatically, and the user self-enrolls from the one-page guide.
Cut over in waves, behind a landing page¶
Build a simple landing page at connect.example.com with per-OS download links, a log-in button,
three steps, and one plain line saying the old VPN still works until a given date. Then migrate by
team. Send a wave its link, confirm each person is connected on the mesh, and only then have them
stop using OpenVPN. Keep the old VPN as the fallback until the wave is confirmed stable, and only
then start the next one.
When the last interactive user is across, the old VPN still doesn’t get deleted. It drops down to carrying site-to-site backup tunnels only. You can revoke individual client certs, but the server stays. This is also where the wider maintenance window fits in. The pieces this project replaced, the old single-purpose VMs, get cleaned up after everyone is confirmed on the new stack, never before.
The rollback that’s always there
At every wave the escape hatch is the same. The user falls back to OpenVPN, which never stopped
working. And if the whole stack ever needed to disappear, it’s docker compose down plus
dropping one firewall rule, with the old VPN untouched underneath. Reversibility isn’t
something you add at the end. It’s a property you protect at every step.
Honest limits, and what’s deliberately next¶
- OpenVPN isn’t gone, and won’t be. It keeps the site-to-site tunnels. This was never about killing OpenVPN, only about getting people off per-user certs.
- MFA still lives in Keycloak for now. The next phase federates the client’s cloud IdP (Entra) into Keycloak and moves MFA authority to Conditional Access, then closes the loop with an onboarding form that provisions identity from a single source and reconciles it down to the directory’s POSIX accounts. That’s a project of its own, and the right call was to not block this migration on it.
- The far-office latency is unchanged. Around 250 ms is physics. The durable fix is regional compute, tracked separately. The mesh didn’t make it worse, but I won’t pretend it made it better.
The one takeaway¶
If there’s one idea worth keeping from this series, it’s the one that made every part safe. Build the new thing alongside the old, keep it reversible, and make it earn trust one wave at a time. The flashy parts are the WireGuard mesh and the SSO. The part that actually mattered, the reason nobody had a bad day, is that the old VPN was still humming the whole time. A boring cutover is the goal, and boring is hard-won.