The PostgreSQL Conference University of Ottawa Ottawa 2013-05-21 2013-05-26 6 20130525 09:00 00:15 09:00 03:00 MRT 256 Tutorial workshop en This talk is best for those who are are familiar with databases and SQL, but want to learn how to move processing from their applications into the database to improve consistency, administration, and performance. Topics covered include advanced SQL features like referential integrity constraints, ANSI joins, views, rules, and triggers. The presentation also explains how to create server-side functions, operators, and custom data types in PostgreSQL. This talk is best for those who are are familiar with databases and SQL, but want to learn how to move processing from their applications into the database to improve consistency, administration, and performance. Topics covered include advanced SQL features like referential integrity constraints, ANSI joins, views, rules, and triggers. The presentation also explains how to create server-side functions, operators, and custom data types in PostgreSQL. Bruce Momjian http://momjian.us/main/presentations/overview.html#data_processing 13:00 03:00 MRT 256 Data types like you never saw them before A Whirlwind Tour of PostgreSQL Data Types Tutorial workshop en What's the great thing about a database? Why, it stores data of course! However, one feature that makes a database useful is the different data types that can be stored in it, and the breadth and sophistication of the data types in PostgreSQL is second-to-none, including some novel data types that do not exist in any other type of database software! This tutorial will take an in-depth look at all of the native data types built right into PostgreSQL as of version 9.2 (with some peeking ahead to 9.3). We will take a look at the functionality provided with each data type, examples of how they are used both in theory and practice, and of course, best practices that demonstrate the robustness of PostgreSQL. Some of the data types that will be covered are: * Date / Time (which the presenters prefer manipulating in PostgreSQL) * Arrays * Geometric Types * JSON / XML * Range Types * UUIDs * Fulltext Search types * Composite and Custom Typing We will also demonstrate some data type extensions that are available to the PostgreSQL. Jim Mlodgenski Jonathan S. Katz 19:00 05:00 L140 hacker2 meet, greet, code, slack Social other en A place to gather... This is the place where many people will gather to work on their laptops, converse, code, hack, slack, and generally behave in cooperative ventures. The times will vary, depending on when people gather. wifi is available, but bring power strips and extension cords to share the wealth. The lounge is located in the [Residence](http://g.co/maps/8scp6), ground floor, just beyond the front desk. Ask if you can't find it. Dan Langille 09:00 03:00 MRT 256 Replication, Backup, and Human Disaster Recovery Tutorial workshop en The ZFS filesystem can be used simplify a number of replication, backup and recovery challenges. In this hands-on workshop, an introduction to the ZFS file system's features and capabilities will be presented, then a walk through of common scenarios where ZFS can save considerable hassle and countless hours when users and/or developers go astray. A VM will be provided for a hands-on experience. Attendees will have access an OmniOS VitualBox(R) image with PostgreSQL installed. The virtual machine will also have PostgreSQL instances configured for file-based log shipping, and streaming log replication. All of the commands needed to complete the following exercises will be supplied. EXERCISES: Replicate a PostgreSQL database to a slave server with zero down time on the master using zfs send. Update the slave server using an incremental ZFS send. Set up timed snapshots of your production data. Spin up a second PostgreSQL instance on a ZFS snapshot of your master or slave database. This lets you set up test or validation instances with virtually no additional disk space required. Perform tests on real data, and if things don't go so well, restore the database to its original state in seconds. Turn the slave server into the master and send updates back to the old master. We will also discuss file-based and streaming log replication, and how these can be used in conjunction with ZFS. If you want to perform the exercises in the workshop, you will need a laptop capable capable of running VirtualBox with 2-4GB allocated to the VM. Keith Paskett VM for hands on experience 13:00 03:00 MRT 256 with Git, Sqitch, and pgTAP Tutorial workshop en Hi, I'm David. I like to write database apps. Just as much as I like to write web apps. (Maybe more!) How? Not by relying on bolted-on, half-baked database integration tools like migrations, I'll tell you that!. Instead, I make extensive use of best-of-breed tools for source control ([Git](http://git-scm.org/), database unit testing ([pgTAP](http://pgtap.org/)), and database change management and deployment ([Sqitch](http://sqitch.org/)). If you'd like to get as much pleasure out of database development as you do application development, join me for this tutorial. We'll develop a sample application using the processes and tools I've come to depend on, and you'll find out whether they might work for you. Either way, I promise it will at least be an amusing use of your time. Are you tired of the poor tools for managaging database schema changes? Are you offended by the requirement that changes committed to your souce code repository can never be changed, ever? Do you want the database interfaces you create to work the way you think they work? If so, join some of your most hard-working, intelligent, and attractive database colleagues for this tutorial, affectionally known as "Theory's Pragmatic approach to Database Development." We'll go through the process of developing, modifying, testing, and deploying a database application for a web service. We'll make extensive use of the awesome tools currently at our disposal to grease the wheels of creation, collaboration, and completion: * [Git](http://git-scm.org/) for source code management. We'll cover how Git thinks, and discover the freedom to hack without annoying your collaborators. * [pgTAP](http://pgtap.org/) for database unit testing. Make sure that the database interfaces you create work the way you think they do -- and stay that way! * [Sqitch](http://sqitch.org/) for database schema change management and deployment. Liberate yourself from the constraints of half-baked, bolted-on migrations and discover the freedom of developing, modifying, and deploying your database like the database maven you are. If you're a database geek who works with source code, applications, and other developers, you won't want to miss this get-together. **Preparation** Please come to this session with the following software installed and ready to go on your laptop: * [PostgreSQL 9.2](http://www.postgresql.org/download/) * [Git](http://git-scm.com) * [Sqitch](http://sqitch.org) * Via [CPAN](http://metacpan.org/release/App-Sqitch): `cpan App::Sqitch` * Via [Homebrew](http://mxcl.github.io/homebrew/): `brew tap theory/sqitch && brew install sqitch_pg` * [pgTAP](http://pgtap.org/) * Via [PGXN](http://pgxn.org/): `pgxn install pgtap` * [Source](http://master.pgxn.org/dist/pgtap/) * [`pg_prove`](https://metacpan.org/module/pg_prove) * Via [CPAN](https://metacpan.org/module/pg_prove): `cpan TAP::Parser::SourceHandler::pgTAP` * [Source](http://cpan.metacpan.org/authors/id/D/DW/DWHEELER/TAP-Parser-SourceHandler-pgTAP-3.29.tar.gz) David E. Wheeler Git Sqitch pgTAP Slides on SpeakerDeck Git Project 19:00 05:00 L140 hacker2 meet, greet, code, slack Social other en A place to gather... This is the place where many people will gather to work on their laptops, converse, code, hack, slack, and generally behave in cooperative ventures. The times will vary, depending on when people gather. wifi is available, but bring power strips and extension cords to share the wealth. The lounge is located in the [Residence](http://g.co/maps/8scp6), ground floor, just beyond the front desk. Ask if you can't find it. Dan Langille 15:00 04:00 Royal Oak registration The social way to register: at the pub Social other en Pick up your registration pack Stop by the Royal Oak Pub on Laurier Street and get your registration pack. You'll help us avoid long line ups on Friday morning and you get to have a beer, and chat with your fellow attendees. We guarantee you'll spot someone famous. Dan Langille 09:00 00:45 MNT 202 keynote The present & the future Plenary lecture en Postgres at Disqus * The present * Where we use postgres, where we don't * Our cluster size * Our workload * Our pain points * Logical replication * Connecting to postgres * The future * Share a story about how using 3rd party nginx modules saved us a huge amount of work & overhead * Draw a comparison to postgres extensions, and the challenge of using the "right tool for the job" * Building an app inside your database vs. building an app on top of your database, unique to postgres world * Postgres extensions we find interesting and their use cases * Postgres extensions we'd like to help build Mike Clarke slides 10:00 01:00 MNT 202 How to manage race conditions in database access DBA lecture en This talk will discuss the problems which can occur when multiple database transactions are accessing the same data in PostgreSQL, and the various approaches to managing those problems. The Multi-Version Concurrency Control (MVCC) used in PostgreSQL allow better performance and concurrency than older Strict 2 Phased Locking (S2PL) techniques, but presents some challenges for handling race conditions. This talk will describe the nature of the problems which can be encountered and discuss alternatives for managing those problems. No one technique is best for all workloads, so the pros and cons of various techniques will be discussed, and benchmarks of various workloads using various techniques will be shown, to illustrate the trade-offs. Kevin Grittner 11:30 01:00 MNT 202 Strategies and Roadmap DBA lecture en This will not be the most exciting talk, nor is there (currently) a simple answer to make hardware corruption problems go away. But it's important -- without being careful, it's easy for corruption to spread to replicas and backups, leaving data hopelessly lost. Or, a strange crash due to corruption could take many engineering resources to analyze. This talk is about kinds of hardware corruption that can and do happen, and the ways to detect and contain the corruption as quickly as possible. Additionally, we'll discuss a roadmap of improvements to postgresql to make this an easier process; as well as alternatives (such as detecting corruption in the filesystem). Note: Some storage systems do provide strong protections against data corruption. This talk is primarily (though not exclusively) targeted at users of the local filesystem, particularly on Linux. These are the topics that will be addressed: 1. Why not deal with this in the filesystem? 2. The different kinds of corruption. 3. When to detect the corruption, and how to contain it. 4. Data page checksums 5. Backups and corruption 6. Replication and corruption 7. Background and offline detection 8. More work to be done Jeff Davis 12:30 01:00 MNT 202 A DEFCON 21 Qualifier Social contest en The Schemaverse is a space-based strategy game implemented entirely within a PostgreSQL database. Compete against other players using raw SQL commands to command your fleet. Or, if your PL/pgSQL-foo is strong, wield it to write AI and have your fleet command itself! Compete in the multi-day tournament at PGCon 2013 for a chance to qualify for the championship tournament, to be held later in the year at DEFCON 21 in Las Vegas. Josh McDougall The Schemaverse 13:30 01:00 MNT 202 If you want to know right now what your server has been doing DBA lecture en When problems arise, every DBA wants to know what the database has been up to. If your best idea so far has been to parse the log files or load the CSV logs into another database, you might want to learn about a better way. With logging hooks, you can attach data sinks directly to the PostgreSQL server log stream and analyze logging events as they happen. Many useful applications arise this way: - put log data into another PostgreSQL database for ad hoc querying - store log data in Hadoop for analytics - throw log data into a graphing system - generate alerts directly from logging events I'll show you how to put these things together and use them in practice. Peter Eisentraut presentation material 15:00 01:00 MNT 202 An analysis of common query planner failure modes Performance lecture en The PostgreSQL query planner does an excellent job with most queries, but no query planner is perfect. In this talk, I'll analyze some of the commonly-reported cases where it fails to produce an acceptable plan, based on posts to pgsql-performance and other cases I've encountered on the job. Robert Haas 16:15 01:00 MNT 202 lightning Short sharp descriptions of short topics Social lightning en A regular feature, PGCon will have a Lightning talks session, with presentations on diverse topics. The format remains essentially the same: in a one hour period, audiences are entertained and informed by a rapid fire series of short talks on interesting new or on-going work by individuals or groups. Slides aer permitted, but not obligatory; pictures are highly recommended. Topic areas include new open source software projects, works in progress for future releases of existing projects, student projects, etc. Lightning talks topics this year may make good conference papers next year! The number of slots is limited, and experience suggests there will be more takers than slots. Sign up well in advance to be assured a spot. Please subscribe to the PGCon announce mailing list and wait for the call to go out for submissions. Magnus Hagander 10:00 01:00 MRT 221 A First Look At What's Coming In Postgres 9.3 9.3 Features lecture en Postgres 9.3 is still a few months out, but we've already got some interesting things to play with. Join us in this overview of new features coming in Postgres 9.3, and see what goodies it has in store for you. There's a ton of stuff lined up for Postgres 9.3, and this is the place to get your first look. We may or may not talk about exciting things like: - performance improvement and memory changes - improved array support - event triggers - lateral support - background workers and much much more. Robert Treat 11:30 01:00 MRT 221 Hacking lecture en Logical Replication in the context of postgres to this date consists out of several independent out-of-core solutions. While some of these solutions are great, the existance of many of those also causes problems like code duplication, lack of trust, features, reliability and peformance. As part of a proposal to include one logical replication solution into core postgresql we submitted the changeset generation/extraction part as a core infrastructure to postgres. In a way its usable by all the existing replication solutions and for lots of other usecases. This talk is about: * the architecture of the committed/proposed changeset generation mechanism (2/3) * An overview over further proposed patches (9.4+) to get a whole logical replication into core postgres (1/3) Andres Freund Talk slides http:// 13:30 01:00 MRT 221 Add your feature to PG! Hacking en We'll cover how to write your first patch to PG, submit it for review, and profit! Ever wished PostgreSQL had a particular capability or feature? Know a bit of C? This talk will walk you through writing a patch for PG, what needs to be modified to add an option to an existing command (grammar, execution, etc) and the major components of PG (parser, commands, memory management, etc). We'll also cover the PG style guidelines, a crash-course on using git, how to submit your patch, and the review process. Stephen Frost 15:00 01:00 MRT 221 9.3 Features workshop en Row-level security has a feature with longstanding development. It can enforce users to reference or modify part of rows according to the configured security policy. So, we can utilize this feature as if virtual private database on other commercial database, however, we designed this feature much carefully to keep both of reliable security and minimum performance trade-off. In addition, we enhanced usability to allow row-level security policy using usual expressions, also plan to integration with label based mandatory access control. This session introduces which was the problematic scenario being called "leaky-view", solutions for them as basis of this feature, row-level security feature being newly supported, and the future plan towards integration with label based mandatory access control. We don't assume audience has deep knowledge on planner. Typical use cases will be helpful for web-application developers who like to ensure reliable separation between users. KaiGai Kohei discussion page for row-level security 10:00 01:00 MRT 256 Applications lecture en We have developed a versatile system to monitor machines and groups of machines. The system is well proven and used in industrial production, power supply industry, research and development. PostgreSQL is used to store all the measurements and metadata. The system also containes some SCADA elements. This talk explains the database related essentials of the monitoring part of the software. Beside configuration, a monitoring system mainly consists of four main functionalities. Solutions to some important topics will be explained. 1. Reading of measurements from devices - Connection of heterogeneous datasources (PLCs, sensors, computer programs, …) implementing different protocols. - Managing network problems between devices and monitoring system. - Consideration of time zones. - Synchronizing of timestamps between connected devices. - Handling of reconfiguration. 2. Storage in PostgreSQL - Efficient database structures to avoid an inflation of tables (no EAV!). - Optimizing the size of the database. - Unified treatment of the simple data types: boolean, integer, floating point numbers. - Working with complex data like text, images or spectra. - Implementing storage strategies and pretriggers to minimize disk space. - Realizing high input rates (>3 million measurements per second over weeks). - Handling large amounts of data. - Preparing the database structure for fast retrieval and statistical evaluations. - Avoiding disk overflow. 3. Fast retrieval - Minimizing dataflow by using dynamic aggregations. - Processing of statistical analytics. - Generating charts and reports. 4. Alarming We see our solution as a new way to implement the basic ideas of a Round-Robin Database. Roland Sonnenschein 11:30 01:00 MRT 256 Applications lecture en Nulls are a very useful but also very error-prone relational database feature. This talk is designed to help applications developers better manage their use of Nulls. It covers the use of NULLs in relational databases, with a focus on Postgres behaviour. It covers three-value logic, comparing nulls, mapping nulls to strings, indexing nulls, and aggregates. Bruce Momjian PDF slides 13:30 01:00 MRT 256 Full Text Search extension Advanced Features lecture en How to integrate both tools and obtain the best performance + reliability. This talk is focused on the new and hottest features on Sphinx (2.1.1 beta) and PostgreSQL. How to combine those tools and HA new features will be showed up during the presentation and also, how to reach a high performance and simple text search Emanuel Calvo 15:00 01:00 MRT 256 Bridging the Divide with Distributed Foreign Tables Big Data lecture en Apache Hadoop is an open-source framework that enables the construction of distributed, data-intensive applications running on clusters of commodity hardware. Building on a foundation initially composed of the MapReduce programming model and Hadoop Distributed Filesystem, in recent years Hadoop has expanded to include applications for data warehousing (Apache Hive), ETL (Apache Pig), and NoSQL column stores (Apache HBase). In this talk we describe recent work done at Citus Data that makes it possible to run a distributed version of PostgreSQL on top of Hadoop in a manner that combines the rich feature set and low-latency responsiveness of PostgreSQL with the scalability and performance characteristics of Hadoop. This talk will begin with a high level overview of Hadoop that focuses on its distributed storage layer and block-based replication model. Next we will look at the data model of the Apache Hive data warehousing system and explain how it enables features such as schema-on-read, support for semi-structured data, and pluggable storage formats. Finally, we will describe how we leveraged these ideas and Foreign Data Wrappers to build a distributed version of PostgreSQL. This version runs natively on Hadoop clusters and seamlessly integrates with other components in the Hadoop ecosystem. Carl Steinbach 19:00 05:00 L140 hacker1 meet, greet, code, slack Social other en A place to gather... This is the place where many people will gather to work on their laptops, converse, code, hack, slack, and generally behave in cooperative ventures. The times will vary, depending on when people gather. wifi is available, but bring power strips and extension cords to share the wealth. The lounge is located in the [Residence](http://g.co/maps/8scp6), ground floor, just beyond the front desk. Ask if you can't find it. Dan Langille 18:00 03:00 Out and about socialouting sponsored by EnterpriseDB Social other en EnterpriseDB invites all PGCon attendees to a big evening with drinks, appetizers, dinner and music on Thursday May 23rd at [My Condo](http://mycondoottawa.ca/), just minutes from the conference venue in the Byward Market. We have exclusive use of My Condo between 6:00 - 9:00 pm, meaning that all 4 floors including the 4th floor patio will be ours to move around, mingle and catch up with fellow attendees. Food stations will be available on multiple floors, BBQ burgers, Jambalaya, Fettuccini Primavera and Honey Mustard Chicken, so come hungry! NOTE: Remember to bring your conference badge with you as it will be your ticket to entry at [My Condo](http://mycondoottawa.ca/). For directions, please consult the [official conference map](http://g.co/maps/rzxqq). MyCondo turns into a night club at 9:30pm, and everyone is welcome to stay free of charge and enjoy the evening - [check it out](http://mycondoottawa.ca/nights/) Dan Langille Map 10:00 01:00 MNT 202 Availability, Durability, Architecture, Replication... explained Scaling Out lecture en How to implement PostgreSQL in a demanding project, what are the different technical offerings good for? All you wanted to know about replication and never dared to ask. PostgreSQL includes several High Availability solution, some replication solutions, and some external Open Source projects complement the offering. When to use which project and what for? This talk will present the usual needs you want to address in a medium size project and how to use several replication solutions to implement them. Dimitri Fontaine link to slides 11:30 01:00 MNT 202 How a 25 Year Old Database Became The Next Big Thing Community lecture en For a long time, Postgres has been the best kept secret of smart DBAs. Lately, it's become well-known that "all the cool kids" are using Postgres. In a recent Ruby on Rails community survey, more developers preferred Postgres in production to MySQL; Django's founder, Jacob Kaplan-Moss, has said given the option he'd only support Postgres. What were the forces that helped bring Postgres into the spotlight? Why are people switching to Postgres in droves (and where are they going when they don't?) In this talk, we'll look back over some of the history of the database world, who the competitors are (and where they're at), and talk about what's coming up for Postgres and what awesome projects are going on in the community to help drive the next 25 years of development. Peter van Hardenberg 13:30 01:00 MNT 202 lecture en Whenever you perform point-in-time-recovery or failover in PostgreSQL, a new timeline is created. In the best case, timelines help you stay sane in complicated recovery situations and keep your database consistent. In the worst case, they will thoroughly confuse you. This talk explains timelines, how they are used by the system, and how you can stay sane with them and even take advantage of them. Version 9.3 brings some improvements to the handling of timelines, these are discussed as well. Heikki Linnakangas 15:00 01:00 MNT 202 Applications lecture en Performance tips and tricks on SQL level. What is the difference between inner, outer, cross and natural join? What is a correlated subselect? What is the advantage and what the disadvantage of an index. Is NOT and OR evil? When do I need normalisation and when denormalisation? All this questions and lots of other performance tips and tricks on SQL level will be given in the talk. Susanne Ebrecht 16:30 01:00 MNT 202 Case Studies lecture en PostgreSQL Enterprise Consortium (PGEcons) is a non-profit organization aiming for promoting PostgreSQL in production use, especially in mission critical area. It is formed by leading IT companies in Japan last year. Currently PGECons has 36 company members. In the first fiscal year, PGECons performed two major technical activities: PostgreSQL scale up/scale out evaluation (by Working Group 1: WG1) and establishing migration process from commercial DBMSs to PostgreSQL (by Working Group 2: WG2). These objectives were determined based on requests from PostgreSQL users and PGECons members and PGECons believes that sharing the experience is a big benefit for PostgreSQL community. In this talk we present current achievements of WG1 and WG2. PostgreSQL Enterprise Consortium (PGEcons) is a non-profit organization aiming for promoting PostgreSQL in production use, especially in mission critical area. PGECons performed two major technical activities conducted by two working groups (WG1 and WG2). In this talk we present current achievements of WG1 and WG2. The tests conducted by WG1 fall into two categories: scale up test and scale out test. For scale up test the target was a 80 physical core server to check how PostgreSQL 9.2 behaves against concurrent transactions. We will report the detailed results including the fact that PostgreSQL scales up to 80 concurrent users with read-only load. For the scale out tests, we tested three configurations: PostgreSQL's cascading replication with a master, a cascading standby, and four standbys, pgpool-II running in native replication mode with 4 nodes and Postgres-XC with 4 data nodes. Each configuration showed different characteristics. For example, pgpool-II performed well with read-only load, while Postgres-XC was strong in write load. We will present them with detailed data. To promote adoptions of PostgreSQL in enterprise domain, WG2, Operation and Design working group in official, examined several related issues on the start time and determined to focus on DBMS migration. According to the survey of attendees at PGECons opening seminar, users of Oracle, DB2, SQL Server, Sybase and MySQL are in total 2.5 times more than those of PostgreSQL, even in such a PostgreSQL event. Because many of those utilizations are predicted to be enterprise ones, dealing with them is indispensable for our purpose. In the first fiscal year, WG2, participated by 11 software businesses, carefully studied the DBMS migration process, divided it into 14 subtasks, and started to tackle to 11 of them by dedicated sub-teams. Selective topics from these activities as well as the big picture of WG2 will be presented in this talk. Hitoshi Hemmi Tatsuo Ishii 17:30 01:00 MNT 202 prizes, auctions, fun, games Plenary other en The Traditional Closing Session Watch the video. We raised thousands for charity. Dan Langille 10:00 01:00 MRT 221 Using native wrappers and a Foreign Data Wrapper for two-way Redis integration Applications lecture en Redis is a high performance in-memory distributed data store, which can work well in conjunction with PostgreSQL in certain types of applications. The talk will introduce native wrappers for the Redis API, which allow pushing data into Redis at high speed, as well as use of both the native API and the Redis Foreign Data Wrapper to pull data from Redis. We will also discuss possible scenarios where use of each of these is a win, and the trade-offs involved. Andrew Dunstan 11:30 01:00 MRT 221 Advanced Features lecture en The Corosync/Pacemaker framework which is part of the LinuxHA stack can be used to provide monitoring and automatic failover capabilities. Now with streaming replication in place, it makes sense to use this robust infrastructure for PostgreSQL master and its replica handling. The open source pgsql resource agent also does this now. This talk will talk about using Corosync/Pacemaker with Postgres. The talk will be on the following lines: * Introduce the Corosync/Pacemaker LinuxHA stack and talk about its features * Talk about PostgreSQL 9 and the high availability requirements * Talk about how streaming replication helps meet these requirements * Talk about how using Virtual IP and the Corosync/Pacemaker resource agents, a robust high availability solution can be provided by using PostgreSQL streaming replication I have done work around high availability in Postgres-XC and blogged about it as well at http://www.stormdb.com/blog/nikhil_sontakke/postgresql Corosync/Pacemaker is a great platform to use for HA and talking about it at PGCon will raise awareness about it. Nikhil Sontakke 13:30 01:00 MRT 221 in realtime Performance lecture en Postgres comes with several introspection tools out of the box. Some are easier to understand than others, but they are all useful. Recent improvements in 9.2's pg_stat_statements make it even easier gain insights into the performance of your application. This talk will explore these built in tools, and what it takes to combine them to provide real-time visualizations of your database. Other topics will incldue - What metrics are the most valuable and how to use them - A deep dive into example application for realtime postgres visibility - Storage of postgres statistics - Tools for time series visualization - Collecting metrics at scale - What works and, just as importantly, what does not work Will Leinweber slides http:// 15:00 01:00 MRT 221 Theory and practice of query progress indication Hacking lecture en Your query has been running for 70 hours. Should you kill it now or ignore angry calls for a few more hours and hope it returns the result? A question many a DBA have asked themselves. This talk will try to cover some of the techniques the database system could use in order to make decisions like that easier. We'll describe an approach based on existing research papers and report on the attempt of implementing it in a useful way inside PostgreSQL. There is ample scientific literature about reporting query progress in relational database systems. Some of the papers published even mention implementations in PostgreSQL. The practicalities, however, are often skimmed over. The talk will start by describing a method for calculating a progress indicator for running queries proposed by Surajit Chaudhuri, Vivek Narasayya, and Ravi Ramamurthy in their 2004 SIGMOD paper. We'll try to see how the terms used in the paper translate to modern PostgreSQL and what practical challenges lie before a hopeful implementer. We'll continue with a demonstration of a module that could be grown into a useful progress indicator solution. The topic will also be an excuse for a little excursion through the PostgreSQL executor and its specific behaviour that needs to be accounted for when calculating query progress. We'll try to give the listeners a basic understanding of how the executor works and familiarize them with nomenclature used in that subsystem. Jan Urbański Estimating Progress of Execution for SQL Queries 16:30 01:00 MRT 221 Nested hstore with arrays support. Hacking lecture en We present a prototype of nested hstore data type with arrays support. We consider the new hstore as a step forward true json data type. Recently, PostgreSQL got json data type, which basically is a string storage with validity checking for stored values and some related functions. To be a real data type, it has to have a binary representation, development of which could be a big project if started from scratch. Hstore is a popular data type, we developed years ago to facilitate working with semi-structured data in PostgreSQL. It is mature and widely used data type with indexing support. Our idea is to extend hstore to be nested (value can be also hstore) data type and add support of arrays, so its binary representation can be shared with json. We present a working prototype of a new hstore data type and discuss some design and implementation issues. Oleg Bartunov Teodor Sigaev 10:00 01:00 MRT 256 Lessons learned from deploying Postgresql at University of Alberta Case Studies lecture en University of Alberta migrated their central Learning Management System from Blackboard Vista on Oracle to Moodle on Postgresql 9.0 running inside VMWare cluster. We went from a pilot project of 13 courses in January 2011 to running all centrally supported courses (3600+) in Moodle in September 2012. Our central Moodle instance has seen more than 500,000 page loads and over 24,000 unique visitors in a single day. Since then we deployed Postgresql in a number of mission critical applications including our ticketing system and Enterprise Document Management system. Over the last two years we have learned a few hard lessons and overcome a few challenges involved in running Postgresql in a 24x7 production environment, although the experience has been predominantly a positive one. We would like to share our experiences (as well as code and documentation) that will help other shops setup their first mission critical Postgresql installation. Specific topics covered: * Configuring postgresql for production workload * What to monitor and when to act on it * Backups, cloning and recovery - omnipitr is your friend * Scalability * Upgrade paths and strategies * Replicating your production load against your data Greg Dostatni Extra content 11:30 01:00 MRT 256 Deployment automation for PostgreSQL DBA lecture en Puppet is a platform for I.T. automation and configuration management that has gained popularity in the devops movement. This talk will provide an introduction to puppet and talk about how Afilias uses puppet to deploy and maintain over one hundred servers running PostgreSQL that support key parts of the DNS infrastructure Puppet allows DBA's to automate the installation and configuration of PostgreSQL across multiple servers. This talk will explain how puppet can be used to install and configure PostgreSQL. Automated configuration of your database servers keeps servers looking the same allowing you to control differences. Managing your database servers through puppet also allows your to easily reproduce your configuration for development and QA purposes. The talk will cover the basics of puppet and best practices for deploying puppet to manage PostgreSQL including: * How to install PostgreSQL via puppet and create database clusters * How to manage postgresql.conf, pg_hba.conf and .pgpass via puppet * Best practices for dealing with configuration differences between different database servers and different environments Afilias is a registry services provider running registries for numerous TLDs including: .INFO, .ORG, .MOBI, .XXX, .IN, .ME, .ASIA, .MOBI. Afilias has been using PostgreSQL since the 7.x days and currently has more 100 production database servers running PostgreSQL. Steve Singer 13:30 01:00 MRT 256 DBA lecture en Rumor has it all these other users and companies do this thing called "backup" on their database. And people keep saying it's a good idea. But what does it actually mean? In this talk we'll go through the different options we have for PostgreSQL backups, how to use them and some good tips and best practices for setting up a backup strategy that actually works when disaster strikes. Magnus Hagander 15:00 01:00 MRT 256 Aka how long a "\dt" takes on a 1B tables database DBA lecture en Usually “large” databases are considered as such for the high number of records they hold, reaching billions or even more than that. But what about creating a billion... tables? Sometime ago, this apparently crazy question was found in a database soup (http://it.toolbox.com/blogs/database-soup/one-billion-tables-or-bust-46270). It may not be your day-to-day task, but the task of creating them exposes some topics about PostgreSQL internals, performance and large databases that may be really worth for your day-to-day. Join us for this talk, where we'll be discussing topics such as catalogue structure and storage requirements, table speed creation, differences between PostgreSQL versions and durability vs. table creation speed tradeoffs, among others. And, of course, how long a “\dt” takes on a 1B tables database :) This talk will explore all the steps taken to achieve such a result, raising questions on topics such as: * The catalogue structure and its storage requirements. * Table creation speed. * Durability tradeoffs to achieve the desired goal. * Strategy to be able to create the 1B tables. Scripts / programs used. * How the database behaves under such a high table count. * Differences in table creation speed and other shortcuts between different PostgreSQL versions. * How the storage media and database memory affects the table creation speed and the feasibility of the task. * If it makes sense to have such a database. It is intended to be a funny, open talk, for a beginner to medium level audience, interested in large databases, performance and PostgreSQL internals. Álvaro Hernández Tortosa 16:30 01:00 MRT 256 Managing Your Thundering Herd DBA lecture en Amazon Web Services provides tremendous tools and techniques for scaling services up and down in response to planned or experienced load. However, too many systems are configured to use AWS as an equipment-rental facility, which wastes money and does not take advantage of AWS' unique properties. We'll talk about how to build systems that flex-scale using AWS tools. Among the topics we'll cover are: -- Designing your application and database for sharding and scaling. -- Planning for load spikes. -- Detecting load fluctuations. -- Scripting your scale-up/scale-down functionality. -- Scaling the database vs scaling the application front-end. -- Monitoring and fault-recovery. The demonstrations will be specifically about AWS, but the techniques can also be applied to other cloud environments. Christophe Pettus 19:00 05:00 L140 hacker3 meet, greet, code, slack Social other en A place to gather... This is the place where many people will gather to work on their laptops, converse, code, hack, slack, and generally behave in cooperative ventures. The times will vary, depending on when people gather. wifi is available, but bring power strips and extension cords to share the wealth. The lounge is located in the [Residence](http://g.co/maps/8scp6), ground floor, just beyond the front desk. Ask if you can't find it. Dan Langille 10:00 01:00 MNT 202 Room #1 Unconference Day lecture en This is where we will decide the content of the rest of the day. Turn up. Tell us what you want to hear. Tell us what you want to talk about. Josh Berkus Details on wiki 11:00 02:00 MNT 202 Room #1 Unconference Day lecture en Turn up here to organize the rest of the day. Those that turn up, get to speak. Josh Berkus Details on wiki 14:00 03:30 MNT 202 Room #1 afternoon Unconference Day lecture en The content of the day will be decided on the day. Those that turn up, get to speak. Josh Berkus Details on wiki 11:00 02:00 MRT 221 Room #2 Unconference Day lecture en The content of the day will be decided on the day. Those that turn up, get to speak. Josh Berkus Details on wiki 14:00 03:30 MRT 221 Room #2 Unconference Day lecture en The content of the day will be decided on the day. Those that turn up, get to speak. Josh Berkus Details on wiki 11:00 02:00 MRT 256 Room #2 Unconference Day lecture en The content of the day will be decided on the day. Those that turn up, get to speak. Josh Berkus Details on wiki 14:00 03:30 MRT 256 Room #2 Unconference Day lecture en The content of the day will be decided on the day. Those that turn up, get to speak. Josh Berkus Details on wiki 09:00 05:00 Out and about tourist See the sights in and/or around Ottawa Social other en Play Tourist We will organize a visit to one of the many local tourist attractions. Dan Langille