PGCon2012 - Final Release

PGCon 2012
The PostgreSQL Conference

Laura Thomson
Day Talks - 2 - Friday - 2012-05-18
Room MRT 218
Start time 11:30
Duration 01:00
ID 492
Event type Lecture
Track Big Data
Language used for presentation English

Moving Day: Migrating Big Data from A to B

In 2011 we moved the Mozilla crash reporting system from old creaky hardware in San Jose to a new shiny datacenter in Phoenix. This system contains more than 40TB of data in HBase, the Hadoop database, and PostgreSQL. The data collecting app has a requirement for close to 100% uptime. On top of that we have data processing, an API, and a webapp. After many months of work, the migration went seamlessly.

In this session we’ll talk about:

  • The checklist manifesto, reprised, and understanding the critical path

  • How to move all that data in a reasonable timeframe

  • The importance of devops culture in success

  • Automating packaging and configuration and how it will save you

  • Understanding the difference between old and new platforms: correctness testing, load testing, and smoke testing

Attendees should walk away with an outline of everything they’ll need to do to achieve a successful data center migration.