PGCon2014 - 20140704

PGCon 2014
The PostgreSQL Conference

Robert Haas
Day Talks - Day 2 - Fri May 23 - 2014-05-23
Room Monpetit 202
Start time 11:00
Duration 00:45
ID 693
Event type Lecture
Track Hacking
Language used for presentation English

Implementing Parallelism in PostgreSQL

Where We Are Today, and What's On The Horizon

PostgreSQL's architecture is based heavily on the idea that each connection is served by a single backend process, but CPU core counts are rising much faster than CPU speeds, and large data sets can't be efficiently processed serially. Adding parallelism to PostgreSQL requires significant architectural changes to many areas of the system, including background workers, shared memory, memory allocation, locking, GUC, transactions, snapshots, and more.

In this talk, I'll give an overview of the changes made to background workers in PostgreSQL 9.4 and the new dynamic shared memory facility, which I believe will form the foundations of parallelism in PostgreSQL, and discuss some lessons I learned while implementing these features. I'll also discuss what I believe is needed next: easy allocation of dynamic shared memory, state sharing between multiple backends, lock manager improvements, and parallel algorithms; and highlight what I believe to be the key challenges in each area.