pg_rewind: Reusing a Diverged Old Primary

The one thing to understand first

When a standby is promoted to primary (after the old primary failed or was isolated), the cluster has a new leader on a new timeline. If the old primary comes back, it has diverged: it may contain transactions that were never replicated, and it is on the old timeline. You cannot simply reconnect it as a standby — its history conflicts with the new primary. Historically the only fix was a full pg_basebackup rebuild, copying the entire database. pg_rewind avoids that.

pg_rewind copies only the data blocks that actually diverged since the split point — not the whole database. It turns a multi-hour rebuild of a failed primary into minutes, by discarding (not recovering) the old primary's unreplicated writes.

What divergence looks like

... common history ...  <- divergence point (LSN)
old primary:                       \--- A, B  (transactions never replicated)
new primary (promoted):            \--- X, Y, Z  (new timeline)

The two servers agree up to the divergence point, then differ. To reuse the old primary, you must undo its divergent changes (A, B) and bring it in line with the new primary so it can stream from it.

How pg_rewind works

pg_rewind (src/bin/pg_rewind/) does the minimum copying needed:

• It finds the divergence point by comparing the two servers' timeline histories — the last LSN they had in common.
• It reads the old primary's WAL from that point forward to identify every data block that changed on the old primary after divergence.
• It copies only those changed blocks (plus changed config/clog files) from the new primary, overwriting the divergent ones on the old primary.
• The old primary's data directory now matches the new primary at the divergence point; it can then replay the new primary's WAL forward via normal streaming.

The win: it transfers only the blocks that actually differ, not the whole database — turning a multi-hour rebuild of a large cluster into minutes.

Prerequisites

• Data checksums enabled, or wal_log_hints = on — pg_rewind needs hint-bit changes to be WAL-logged so it can detect all modified blocks. Without one of these it refuses to run.
• Access to the new primary (a connection or its data directory) to fetch the authoritative blocks.
• The old primary must be cleanly shut down first so its WAL is consistent.

Layer 3 — Watch it happen on your own database

After a clean shutdown of the old primary, one command resynchronises it against the new leader:

# On the old primary (now to become a standby), after a clean stop:
pg_rewind \
  --target-pgdata=/data \
  --source-server='host=new-primary user=replicator dbname=postgres' \
  --progress

# Then configure it to stream from the new primary and start it.
# It comes up as a standby on the correct timeline.

The --progress output shows it transferring only the changed blocks — a tiny fraction of a large database. You can confirm the prerequisite first with SHOW wal_log_hints; or by checking pg_controldata for data checksums; without one of those, pg_rewind refuses to run.

Layer 4 — The levers this hands you

pg_rewind is sharp-edged — its prerequisites and limits are the levers you must set before a failover:

• **Enable wal_log_hints (or data checksums) up front.** pg_rewind needs hint-bit changes WAL-logged to detect all modified blocks; you cannot turn this on retroactively without a rewrite/restart. Decide this at cluster build time.
• It discards the old primary's divergent writes (A, B in the diagram) — they are lost, not recovered. This is exactly why fencing (preventing a demoted primary from accepting writes) matters: anything it accepted after the split and never replicated is gone.
• It is a resynchronisation tool, not a backup tool — only for rejoining a known cluster.
• For very large divergence (old primary far ahead), a fresh base backup may still be simpler; pg_rewind wins when divergence is small.
• It is the standard failover lifecycle step. Patroni and repmgr call pg_rewind to reattach a failed primary as a standby after promotion — promote standby, fence old primary, pg_rewind it back — keeping spare capacity online quickly instead of rebuilding.

Layer 5 — What an Oracle DBA should expect vs what they get

pg_rewind has a very close Data Guard analogue, which makes it one of the more familiar HA tools for an Oracle DBA:

• **pg_rewind ≈ Data Guard's FLASHBACK DATABASE + reinstate of a failed primary,** or RMAN incremental "recover from SCN" — it brings a diverged old primary back into the configuration by copying only the changed blocks.
• The divergence point ≈ the failover SCN; the new timeline ≈ a new incarnation. pg_rewind finds the last common LSN exactly as Oracle reinstatement finds the divergence SCN.
• Discarded divergent writes are the same hazard as reinstating a former primary without Flashback's protection — anything it committed after the split and never shipped is lost, so fencing is mandatory in both worlds.
• No Flashback logs required. Where Oracle reinstatement leans on flashback logs, pg_rewind instead needs wal_log_hints/checksums enabled in advance — the Postgres-specific precondition to set at build time.

Key takeaway

pg_rewind rejoins a diverged old primary as a standby by copying only the blocks that changed since the timeline split, turning hours of rebuild into minutes — but it discards the old primary's unreplicated writes, so fencing is essential. Enable wal_log_hints or data checksums at cluster build time, because you cannot add them retroactively when you suddenly need a rewind.