PGCon2016 - 20180510

PGCon 2016
The PostgreSQL Conference

Jan Wieck
Day Talks - Day 2 - 2016-05-20
Room DMS 1160
Start time 16:00
Duration 00:45
ID 937
Event type Lecture
Track Performance
Language used for presentation English

Benchmarking Databases

Performance Evaluation

Database migration projects are expensive. Be it a change in hardware architecture, operating system, the database system itself or all of it. An important step at the very beginning of such a project is to evaluate if and how well the proposed new system compares to the old one with respect to performance and scalability.

In the absence of a performance test harness for the real application users often try to use "standard" benchmarks to answer that question. However, do those benchmarks really answer "that" question and how closely are various open source benchmarking tools to the standard, they claim to resemble closely?

In this talk I will use mostly performance graphs to discuss

  1. What are the differences to the standard for several open source implementations of the still popular TPC-C benchmark?
  2. How do those differences affect the meaning of the reported performance numbers?
  3. What besides the reported numbers are important data points to monitor to get a better picture?