PGCon2016 - 1.1.35

PGCon 2016
The PostgreSQL Conference

Marco Slot
Day Talks - Day 1 - 2016-05-19
Room DMS 1130
Start time 13:00
Duration 00:45
ID 925
Event type Lecture
Track Scaling Out
Language used for presentation English

pg_paxos: Paxos-based table replication

Bringing the magic of distributed consensus to PostgreSQL

This talk will discuss and demonstrate the pg_paxos extension, which adds an implementation of the Paxos distributed consensus algorithm to PostgreSQL.

pgpaxos provides yet another way of doing table replication in PostgreSQL by automatically replicating a query log through Multi-Paxos. Unlike existing replication solutions, pgpaxos can provide consistent, fault-tolerant, multi-master replication across PostgreSQL servers, at the cost of high read- and write latencies and low throughput. pg_paxos is not a general replication solution, but is especially suitable for applications such as automated fail-over, distributed locking, and managing cluster membership, and can potentially replace components such as Zookeeper.

pgpaxos is implemented primarily in PL/pgSQL, which seems odd, but has two significant advantages: 1) Paxos can be difficult to implement because nodes need to keep their local state in a durable data store. In PL/pgSQL ACID transactions come for free, making the code simple and easy to understand. 2) The core parts of pgpaxos can run in restricted environments such as Heroku and Amazon RDS.

This talk will attempt to explain the Paxos algorithm and the Multi-Paxos technique and why they are not as hard as they are often portrayed. We will discuss the internals of pgpaxos and several applications. Finally, we will show how we deployed pgpaxos to implement a distributed cron service and give a live demo.