Moving Day: Migrating Big Data from A to B

In 2011 we moved the Mozilla crash reporting system from old creaky hardware in San Jose to a new shiny datacenter in Phoenix. This system contains more than 40TB of data in HBase, the Hadoop database, and PostgreSQL. The data collecting app has a requirement for close to 100% uptime. On top of that we have data processing, an API, and a webapp. After many months of work, the migration went seamlessly.

In this session we’ll talk about:

• The checklist manifesto, reprised, and understanding the critical path

• How to move all that data in a reasonable timeframe

• The importance of devops culture in success

• Automating packaging and configuration and how it will save you

• Understanding the difference between old and new platforms: correctness testing, load testing, and smoke testing

Attendees should walk away with an outline of everything they’ll need to do to achieve a successful data center migration.