Using Paxos for consensus among safekeepers.

[phoenix24]

I ask this question without a lot of context as an external observer, if would appreciate if you point me to something where it's already been answered.

safekeeper in it's current implementation rely on paxos for consensus, I've been wondering if raft was considered for the use-case and if so what was the thought process behind it choosing paxos instead.

[arssher]

Current implementation is actually very close to Raft, readme needs an update. The most noticeable difference is the separation of leader, mirroring the general idea of storage separation -- safekeepers only receive records and passively persist it, while stateless compute (Postgres) generates them, acting as a leader. Original Raft paper doesn't mention this idea; on the contrary, in Paxos world, historically roles were as separate as possible -- for example, in Paxos Made Moderately Complex paper you can find even 4 actors (replica, leader from scout + commander, acceptor). Practically always proposers (who proposes records/values) and acceptors (who persist them) are different entities, and that's what we needed here, so we tended to describe things in Paxos terms in the beginning.

As you probably know, there is no single widely accepted Multi-Paxos description detailed enough to be close to real world implementations. It is more generic and Raft can be considered just a variant of Multi-Paxos with certain features. To me, thе key of them is that 1) in Raft log is monotonic, while generic Multi-Paxos runs Synod protocol over each slot (record) almost independently. From this stems the possibility of directly truncating nodes logs to the log of leader; 2) In Raft leader doesn't restamp its term on the older (not committed on the moment of election) entries, from this stems 'can't commit entries from previous terms without generating current term record' rule. And we are like Raft here (though 2) is somewhat reworked to our realms).

[kinthaiofficial]

The Paxos vs Raft question for safekeeper consensus is a classic distributed systems tradeoff — and Neon's write path makes it particularly interesting because the safekeepers aren't a general-purpose replicated state machine; they're purpose-built for WAL durability.

A few observations on using Paxos here:

Multi-Paxos advantage: Once a stable leader is established, steady-state throughput is similar to Raft. The difference shows up in leader elections — Paxos's prepare/promise phase can be more expensive, but for Neon's use case (rare failovers, sustained write workloads) this matters less.

The real question is reconfiguration: Adding/removing safekeepers requires careful membership change handling. Raft has well-studied joint consensus for this. Paxos reconfiguration is less standardized — you'd need to define your own protocol or use something like Vertical Paxos.

Epoch-based approach (what Neon uses): The epoch acts like a ballot number, making it closer to Multi-Paxos than classic 2PC. Each epoch has exactly one proposer (the compute), which simplifies the protocol significantly.

For a write-ahead log replication use case where the proposer is always the primary compute node, the distinction between Paxos and Raft becomes less meaningful — you're essentially implementing a single-proposer protocol either way.

What specific failure modes or performance characteristics are you trying to optimize for?