The PostgreSQL Conference University of Ottawa Ottawa 2011-05-17 2011-05-21 5 Final (2011.06.11) 09:00 00:30 09:00 03:00 DMS 1120 Database Introspection with PostgreSQL DBA workshop en The PostgreSQL system catalogs store metadata about SQL objects such as tables, columns, functions, schemas, and more. This talk will explore the organization and structure of the system catalogs and explain how you can use these catalogs to manage and tune your database. We'll cover applications such as (1) getting a list of all tables that have no indices, (2) determining table and database sizes on disk, and (3) monitoring table, session, and background writer activity, as well as the internal structure of the catalogs and how they're used by the database system. Robert Haas 13:00 03:00 DMS 1120 Keeping yourself out of the pit Performance workshop en PostgreSQL is a database system that can deliver excellent performance for a wide variety of applications. But it's easy to run into an issue that keeps you from seeing its full potential. There are a few basic PostgreSQL configuration and use misunderstandings that cause most of the early performance issues administrators and developers encounter. This talk covers the most common performance problems seen in real-world PostgreSQL deployment. It briefly summarizes each and points out the basics of how to avoid them. It's like navigating a minefield: you'll progress though a lot faster if you can start by knowing what the mines look like. With PostgreSQL 9.0 featuring all new built-in replication and query distribution features, the timing has never been better to consider what a serious jump into using PostgreSQL would look like. Getting the performance related aspects right can be one of the most stressful parts of such a migration, particularly if you're new to the database and its quirks. Follow the suggestions here, and you can build your database to postpone many of these slowdowns until you've already proven your application is a success. Greg Smith 09:00 03:00 DMS 1120 Or, how I stopped worrying and learned to love the ORM DBA workshop en Object-Relational Mappers (ORMs) are the primary interface that application programmers now use to interact with SQL databases. Any DBA can list the problems that they create, but how can we move on to solutions that don't involve a complete reengineering the ORM-database interaction paradigm? Rather than simply declaim that ORMs are bad, let's find a way of mitigating the problems that existing ORMs create. The database community seems to have been blindsided by the rise of Object-Relational Mappers (ORMs) as the primary database interface technology, even though ORMs themselves have been around for years (the core concepts date to the early 1990s and before). The problems with ORMs are real: Poor-quality SQL, schemas that are not well-matched to the capabilities of the database, and database access patterns that are hugely inefficient. Yet ORMs are here to stay, and we must deal with ORMs that currently exist, rather than the theoretical ORMs we wish we had. As database architects and engineers, ORMs are now part of our client universe, and we need to understand them and find ways of working with, rather than against them. We'll talk about why ORMs have the characteristics that they do, and why they appeal so strongly to application programmers over direct-SQL model. Using real-life ORMs, we'll explore ways that database architects can guide the development of ORM-based applications in a way that mitigate the problems that an ORM can create in database interaction. These include ORM-friendly changes to the application, ORM-invisible changes to the database, and (when required) spot alternations to the application to avoid disasters. Christophe Pettus 13:00 03:00 DMS 1120 RDBMS in the social networks age Advanced Features workshop en Despite the NoSQL movement trying to flag traditional databases as a dying breed, the RDBMS keeps evolving and adding new powerful weapons to its arsenal. In this talk we'll explore Common Table Expressions (SQL-99) and how SQL handles recursion, breaking the bi-dimensional barriers and paving the way to more complex data structures like trees and graphs, and how we can replicate features from social networks and recommendation systems. We'll also have a look at window functions (SQL:2003) and the advanced reporting features they make finally possible. The first part of this talk will cover several different techniques to model a tree data structure into a relational database: parent-child (adjacency list) model, materialized path, nested sets, nested intervals, hybrid models, Common Table Expressions. Then we'll move one step forward and see how we can model a more complex data structure, i.e. a graph, with concrete examples from today's websites. Starting from real-world examples of social networks' and recommendation systems' features, and with the help of some graph theory, this talk will explain how to represent and traverse a graph in the database. Finally, we will take a look at Window Functions and how they can be useful for data analytics and simple inline aggregations, among other things. All the examples have been tested on PostgreSQL >= 8.4 Lorenzo Alberton Trees in the database - Slides Graphs in the database - Slides 15:00 07:00 Royal Oak registration The social way to register: at the pub other en Pick up your registration pack Stop by the Royal Oak Pub and get your registration pack. You'll help us avoid long line ups on Friday morning and you get to have a beer, and chat with your fellow attendees. We guarantee you'll spot someone famous. Dan Langille 10:00 01:00 DMS 1140 Scaling Out Most people have come to understand simple replication based horizontal scale-out, but while this technique works pretty well for some read loads, there are system where either the amount of data involved is too great for a single system, or the write based load goes beyond the capacity of a single machine. In these cases we must turn to complex systems of federated databases to build database systems that can sustain perpetual growth. "Sharding" is a buzz word losing some of it's buzz, and that's a good thing. For many people building read-based scaleout systems is all they will ever need, but if it isn't where do you turn? In this talk I'll focus specifically on large scale, application based scale out being used by some of todays largest internet infrastructures. We'll focus on understanding the pattern underlying these systems, how they are implemented (including discussions on MySQL vs. Postgres as a backing store), and how to manage the infrastructure once you've built it. It isn't for the faint of heart, but if you plan to build large scale systems on commodity hardware, you need to know what has worked for others in the past. Robert Treat 11:30 01:00 DMS 1140 How to Review Patches for PostgreSQL Hacking lecture Patch reviewing is an important part of moving PostgreSQL forward. It's critical for anyone submitting patches to PostgreSQL to also understand how to review patches, too. This talk will go through the steps of patch review, including the most important piece- code review. PostgreSQL, more than most applications, must be stable and reliable. Crashes are just not acceptable, ever. Data loss must be prevented. Add to this the many complexities of meeting the requirements of ACID and SQL, and it's clear that the code in PostgreSQL needs to be reviewed very carefully. We depend on the committers to do final review and commit, but they are a very finite resource. The PostgreSQL community has an excellent approach to scaling up our committer resources- make sure what they're reviewing is of good quality. We'll talk about how to become a reviewer, how to use the CommitFest application, what makes a good pre-committer review, some tricks to doing reviews, and what to look for when doing a code review. Stephen Frost 13:30 01:00 DMS 1140 Advanced Features lecture en Federation Database System solves transparent access to heterogeneous DBMS to realize Optimized Database schema design according to optimized business process. Our database team develops Real Federation Database System leveraging PostgreSQL FDW. This federation system enhances the current optimization of Foreign Scan and adds Foreign Data Wrapper for another RDBMS such as Oracle and Microsoft SQL Server. We implement Federation Database System using PostgreSQL FDW and realize following things: - Access heterogeneous external data source as local data - Location, Vendor and Platform Transparency - Without change of existing system change - Real-time Data Reference and Data Update These features give us various advantages as follows: - Optimization of Business Process across multiple business systems - Integrated Database access - Avoidance of redundant data management My presentation explains the development technique for Federation Database System based on PostgreSQL FDW. It includes the optimization technique for heterogeneous distributed database access and integration technique of Foreign Data Wrapper to heterogeneous RDBMS (e.g. Oracle and Microsoft SQL Server) and so on. And also we would like to discuss the requirements for new Federation Database System for this cloud era. Yotaro Nakayama 15:00 01:00 DMS 1140 Hacking lecture en In this session we take a look at the Light Weight Lock facility in PostgreSQL and the various lwlocks. PostgreSQL uses Light Weight Locks to manage PostgreSQL Locks. In this session we take * Describe the framework of PostgreSQL LWLocks facility -> Architecture of the LWLocks Framework -> Lifecycle of an LWLock using its API -> Understanding Exclusive and Shared modes of lwlock -> Internals of LWLock Wait List -> Ways to monitors the events related to LWLocks * Top LWLocks. -> WALInsertLock -> WALWriteLock -> ProcArrayLock -> SInvalReadLock -> CLogControlLock -> DynamicLWLocks Once you start monitoring LWLocks, you will probably find one of the above locks as highly used or contended. We go a bit further in understanding what this lock means and the corrective actions to take to reduce the stress on the locks. Jignesh K. Shah Presentation Slides 16:30 01:00 DMS 1140 How to get the most out of your activity log DBA lecture en Logs can tell you a lot about what's going on in your database: which user logged in when, what queries they ran and how long they took, when the database restarted. We'll walk through the various logging configuration options PostgreSQL offers to help you figure out how to get what you need in a format that's useful to you. We'll also talk about what to do with your logs once you have them (hint: you don't have to stay up at night reading through them!) and, time permitting, wrap up with a group discussion. Logging is an important part of your database monitoring suite, providing historical information for troubleshooting and forensic purposes: which user logged in when, what queries they ran and how long they took, when the database restarted. PostgreSQL offers different file format, message format, and message content options for logging. These options can be kind of overwhelming for the first-time user. We'll walk through the various configuration parameters to help you figure out how to get what you need in a format that's useful to you. Then we'll discuss some solutions (pgFouine and friends) for extracting information from your logs. If there's time, I really like to have a group discussion at the end of my talks - so bring your configs, logs, and war stories! Gabrielle Roth 09:00 01:00 DMS 1160 keynote Should you be Worried? Plenary lecture en Paul Ramsey has been working with geospatial software for more than 10 years, as programmer and consultant. He co-founded the PostGIS spatial database project in 2001, and is currently an active developer and member of the project steering committee. In 2008, Paul received the Sol Katz Award for achievement in open source geospatial software. Geolocation is a hot topic in technology right now, so this topic will offer a tour of the terrain: what geolocation is being used for, where PostGIS came from, how it uses PostgreSQL, who is using it, and the technology challenges that PostGIS (and by extension PostgreSQL) face in the future as use cases continue to push more and more data through the system. Paul Ramsey 10:00 01:00 DMS 1160 What's in the upcoming version 9.1 Features SQL/MED, Sync Rep, SSI, Streaming Backups, SE-PostgreSQL, Writeable CTEs, Range Types, per-Column Collations, EXTENSIONs, directory archives, checkpoint goggles, VIEW triggers, extensible ENUMs, and more. What are these things? Why should they matter to you? Which of them are in 9.1 and which aren't? Josh Berkus 11:30 01:00 DMS 1160 Migrating Oracle databases to PostgresQL Case Studies lecture en Ora2Pg is a GPLv3 licensed tool used to help migrate an Oracle database to a PostgreSQL compatible database. Ora2Pg can be used from reverse engineering Oracle database to huge enterprise database migration or simply to replicate some Oracle data into a PostgreSQL database. This talk by the author of Ora2Pg will start with an introduction to the project and its history. We will continue with the best practices using Ora2Pg for Oracle to PostgreSQL migration. We will cover much of the functionalities coming in the v7.0 release which contains significant speed improvement and new features. An overview of the future development of Ora2Pg will be provided. The talk will also cover the basic techniques and gotchas around migrating from Oracle to PostgreSQL. Gurjeet Singh Home page of the project 13:30 01:00 DMS 1160 bigpostgres 10 Things to Watch Out For When PostgresSQL Gets Big Performance lecture en Size can creep up on you. Some day you may wake up to a multi-terabyte Postgres system handling over 3000 tps staring you down. Learn the best ways to manage these systems as they grow, and find out what new features in 9.0 have made life easier for administrators and application developers working with big data. Size can creep up on you. Some day you may wake up to a multi-terabyte Postgres system handling over 3000 tps staring you down. Learn the best ways to manage these systems as they grow, and find out what new features in 9.0 have made life easier for administrators and application developers working with big data. This talk will lead you through solutions to problems Postgres faces when it gets big: backups, transaction wraparound, bloat, huge catalogs and upgrades. You need to monitor the right things, find the gems in DBA-friendly database functions and catalog tables, and know the right places to look to spot problems early. We’ll also go over monitoring best practices and open source tools to get the job done. Working with multiple versions of Postgres back to version 8.2 will be included, and as well as tips on making the most out of new features in 9.0. War stories will be taken from real-world work with Emma, an email marketing company with a few large databases. Selena Deckelmann http:// 15:00 01:00 DMS 1160 Running PostgreSQL Stored Procedures on a GPU Advanced Features lecture en 2D and 3D Image data is increasing important. New 3D scanning devices are being used to map a variety of large and small spaces and objects. Large GIS data sets are being acquired with new scanners. Computation photography allows the manipulation of images for many applications both scientific and social. GPUs are increasingly being used to accelerate image based computations. This presentation illustrates how PostgreSQL can store and process images using GPU hosted stored procedures, written in OpenCL. This is an outline of my proposed slides • Title • Speaker’s Bio o Brief biographical details of the speaker. • Talk Outline o Overview the talks topics • Parallel Universe o Provides an over view of what GPU (highly parallel chips) will bring to the DBMS world. • The power of GPU’s o Describes the enormous computing power available from GPUs. • OpenCL GPU/CPU Programming Language o Describes the new OpenCL programming language that can be used for parallel programming for CPUs and GPUs. • PgOpenCL a new PostgeSQL Procedural Language o Describes PgOpenCL a new PostgreSQL procedure language that allows stored procedures to be written in OpenCL. • Image Data Types o Outlines 2D,and 3D image data types that are used to storage and processing of images. • Parallel Image searching o Outlines the type of image filter functions that can be used to process images and extract information as SQL tables from images. • Future Directions o Describes the future direction of parallel processing in databases. • Summary o Summarized the main points of the talk • Q &A o Time to answer question from the audience. Tim Child PgCon 2011 Slides http:// http:// 16:30 01:00 DMS 1160 DBA lecture en This talk explores the ways attackers with no authorized database access can steal Postgres passwords, see database queries and results, and even intercept database sessions and return false data. Postgres supports features to eliminate all of these threats, but administrators must understand the attack vulnerabilities to protect against them. This talk covers all known Postgres external attack methods. Bruce Momjian 17:30 01:00 DMS 1160 lightning Short sharp descriptions of short topics lightning en A regular feature, PGCon will have a Lightning talks session, with presentations on diverse topics. The format remains essentially the same: in a one hour period, audiences are entertained and informed by a rapid fire series of short talks on interesting new or on-going work by individuals or groups. Slides aer permitted, but not obligatory; pictures are highly recommended. Topic areas include new open source software projects, works in progress for future releases of existing projects, student projects, etc. Lightning talks topics this year may make good conference papers next year! The number of slots is limited, and experience suggests there will be more takers than slots. Sign up well in advance to be assured a spot. The session chair this year is Selena Deckelmann. Magnus Hagander Selena Deckelmann List of talks + slides 10:00 01:00 DMS 1120 Applications lecture en In todays world, nobody (should) deploy a web application facing the internet without having a proper caching system in place. There are many different solutions to choose from, from manual use of memcached through framework integrated caching to external caches like Squid or Varnish. Most modern frameworks come with integrated functionality for at least one of these methods, and often more than one. However, they often relies on all traffic going through the same framework to work properly - a caching layer in Rails is hard to share with one in Django. This talk will show a way to break the design rules of these frameworks just a little, and have the database help solve this problem. This talk will use a small application written in Python using Django to illustrate the examples, but the method is language independent. Unsurprisingly, the database used is PostgreSQL. Magnus Hagander 11:30 02:00 DMS 1120 Advanced Features lecture en Multi-Version Concurrency Control (MVCC) allows Postgres to offer high concurrency even during significant database read/write activity. MVCC specifically offers behavior where "readers never block writers, and writers never block readers". This talk explains how MVCC is implemented in Postgres and highlights optimizations which minimize the downsides of MVCC. This talk is for advanced users. This is a two hour talk which extends into lunch. There will be a short break to allow people to grab lunch and bring it back into the lecture hall. Bruce Momjian 13:30 01:00 DMS 1120 In short: -1 from me. regards, tom lane Hacking lecture en Writing a PostgreSQL patch can be difficult, and getting it accepted can require even more work than writing it in the first place. Many people contribute to PostgreSQL each year, and these contributions are critical to PostgreSQL's ongoing success and continued improvement. Patches are also regularly turned away for a variety of reasons. In this talk, I'll discuss how to maximize your chances of a successful contribution and what to do next if your patch is rejected. Robert Haas 15:00 01:00 DMS 1120 Introduce PostgreSQL into reliable and large-scale telecommunication support system Case Studies lecture en We will present how to introduce PostgreSQL into NTT’s telecommunication systems. To inform the merit of PostgreSQL for IT system managers we evaluated its performance and showed applicable field of our production systems. To broaden the field we have resolved performance problems like as performance glitch at checkpoints and improved its usability with home-made tools such as pg_bulkload, pg_statsinfo and so on. All these efforts and their results are worth being shared with who plan and proceed PostgreSQL’s enterprise use. NTT group, which is the largest telecommunication career in Japan providing more than 120 million subscribers, has made effort introducing PostgreSQL to its production systems that support telecommunication. Such systems require as same performance and reliability as proprietary DBMSes provide, which shows PostgreSQL should be enhanced in many aspects for enterprise use. We will present what kind of performance characters were required from production systems engineers and how we showed them. At first we evaluate PostgreSQL performance under TPC-C and TPC-W like workload to show its throughputs are enough for many systems. At the same time we found instability of response time and complex maintenance operation like VACUUM, which we thought be large obstacles for introduction. We tried to resolve them and they are finally cleared up by PostgreSQL 8.3. From the view point of usability, our production engineers pointed PostgreSQL lacked some essential utility tool such as fast data-loader and performance monitor. So we provided original open-source tools pg_bulkload and pg_statsinfo to meet such requirements. Finally we will present improvement availability by PostgreSQL 9.0 and show expected applied systems that the feature should enable. Tetsuo Sakata 16:30 01:00 DMS 1120 Hacking lecture en What does the planner do and why does it do it? How can it be improved? This is a follow-on to Robert Haas's planner talk at PGCon 2010. I plan to focus more on why and how the planner works, ie, present a developer's-eye view instead of a user's-eye view of what it does. Topics will include the overall layout of the planner code, preprocessing, join search order, and cost estimation. Hopefully attendees will come away encouraged to hack on the planner and help improve it. Tom Lane 18:30 02:00 The Velvet Room socialouting sponsored by EnterpriseDB Social other en Come and join us for the major evening social event of PGCon 2011! EnterpriseDB is sponsoring this event for all PGCon 2011 attendees. Dinner and drinks will be provided. See the map on the website for directions to the venue. The Velvet Room 62 York St Ottawa, ON K1N 8Y3 6:30pm-8:30pm Dan Langille Map 10:00 01:00 DMS 1140 How to develop an Android App for PostgreSQL Applications workshop en This is a presentation on how-to develop a simple Android application that uses the PostgreSQL JDBC driver to communicate directly with a database. Use the Android SDK with Eclipse to build and deploy an application. Familiarity with Java programming and XML recommended. Learn how to use the Android SDK and Eclipse to develop a simple user interface to connect to a PostgreSQL database and execute simple SQL statements. Programming tasks include: * Setting up a development environment * Executing simple SQL statements * Executing SQL statements with JDBC Prepared Statements and bind variables * Using cursors to handle potentially large volumes of data * Packaging and distributing Android applications Mark Wong 11:30 01:00 DMS 1140 DBA lecture en OmniPITR is a set of scripts designed to help manage and maintain WAL files for Postgres database. It was created in an effort to replace the myriad scripts we've encountered, and written, managing our clients Postgres installs over the years. It includes tools for both archiving, restoration, and creating portable backups on both the master and slave instance. Ever since the advent of PITR in Postgres 8.1, people have been creating 1 off scripts designed to manage PITR and the xlog files that come with it. We've been just as guilty of this as anyone else, making specialized script after script for clients, tweaking for one difference or another. Finally last year we decided we could do better, and OmniPITR was born. OmniPITR is a set of scripts designed to help manage and maintain WAL files for Postgres database. It was created in an effort to replace the myriad scripts we've encountered, and written, managing our clients Postgres installs over the years. It includes tools for both archiving, restoration, and creating portable backups on both the master and slave instance. Designed to work with Postgres 8.2 and above, we've now replaced a whole mish-mash of scripts with something far more standard and re-usable. Cross platform and Postgres licensed, we have started to see more and more adoption outside of our client base, so we feel comfortable that it's a stable option for many of todays most demanding environments. Robert Treat OmniPITR source 13:30 01:00 DMS 1140 Applications lecture en We introduce and describe two modules that grew from the need to perform integrated and efficient Bioinformatics tasks in PostgreSQL: PostBio, a set of methods to store and query genomic sequences and features, and PostStat, a collection of statistical functions that allow for integrated statistical tests. A few practical examples will be presented to showcase the modules. PostBio includes three data types: a GiST-indexable integer interval used to represent biological sequence features; a suffix tree type to search for maximum unique matches; and a compressed suffix array for fast short exact matches. In addition, PostBio provides a set of utilitary sequence routines. PostStat comprises routines that compute a number of cumulative probability distributions, linear regression, and statistical tests, both parametric and non-parametric; the main motivation is to provide a way to test statistical hypothesis in simple models. Luis Carvalho 15:00 01:00 DMS 1140 Reaching peak write performance under heavy loads Performance lecture en One of the most difficult things to do with a database system is handle a high volume of writes, whether it's from a single user or thousands. The steps a database like PostgreSQL takes to protect your data are expensive, and it's hard to split them into independent pieces. Recent changes available in the upcoming PostgreSQL 9.1 have given new insight into exactly how PostgreSQL systems that write constantly break down under serious loads. The replication features in PostgreSQL 9.0 make it easier than ever to scale your application performance up by adding more read-only nodes with additional copies of the data. But that also makes it easier than ever before to end up running into a harder to resolve write bottleneck as a result. This talk will cover the major approaches and trade-offs in tuning for high write performance on a PostgreSQL server. The write-ahead log, checkpoint timing, and buffer management policies of the database all interact to limit just fast data can be shuffled in and out of the server. Some recently added features to PostgreSQL 9.1 will also be introduced. These give better insight than was ever available before into exactly what caused the kind of checkpoint write "spikes" that have traditionally plagued write-heavy PostgreSQL databases. The trade-offs between optimizing for latency and throughput are highlighted when watching operations at this level. Understanding these problems can help you make the right long-term choices about how to tune system, or provide ammunition for why you need to plan an upgrade to 9.1 when it becomes available in the future. Greg Smith 16:30 01:00 DMS 1140 Case Studies lecture en A specific example of using PostgreSQL in enterprise level OSS system for largest pan-russian cellular provider - technology choice reasoning, implementation, maintenance and performance issues. Solution overview and implementation history. Pessimistic and critical comparison PostgreSQL vs commercial database servers, reasons for choosing PostgreSQL. Expected and unexpected problems using Postgres in the field of telecommunications. Solutions and review of still unsolved difficulties. Postgres specific benefits. All these issues are described in the context of database administration and software development, as well as of interaction with the customer and the vendor. Ilya Kosmodemiansky 10:00 01:00 DMS 1160 Range Types and Temporal Data Advanced Features lecture en Range Types are a new feature that captures ranges -- ranges of time, ranges of numbers, ranges of dates -- and gives users the ability to create their own range types. Using Range Types improves performance by using range indexes, simplifies your queries and schema, and enables advanced features such as using Exclusion Constraints to prevent overlap. Most organizations need to handle range types in one way or another. Ranges of numbers or IP addresses are useful; and of course, ranges of time are perhaps the most common. Explicitly telling PostgreSQL that you are working with ranges -- rather than simulating ranges with a "start" and "end" column -- offers major benefits for functionality (e.g. the ability to prevent overlap), performance (GiST indexing), and ease of use. We'll walk through some basic examples, compare with the "two-column approach" (that is, using a "beginning" and an "end" column), show the benefits of range types using temporal data as an example, and give a brief overview of how to define your own range types. Jeff Davis 11:30 01:00 DMS 1160 Space-Partitioning trees in PostgreSQL GIS lecture en We want to present a prototype of new indexing infrastructure, called SP-GiST, which greatly extends current extendability of PostgreSQL to support space-partitioning trees, as well as a family of digital trees like suffix tree, patricia. We'll present programming interface, examples of realizations of some useful datatypes. Also, we'd like to discuss some implementation issues and roadmap for inclusion to PostgreSQL. PostgreSQL is known as an extendable database, which means that other than built-in data types can be developed by experts in specific domain, not core programmers. Currently, there are GiST (Generalized Search Tree) and GIN (Generalized Inverted Index) infrastructure in PostgreSQL, which provides extendability with indexing, concurrency and recovery support for user-defined data types. However, these infrastructures are not enough to support a wide class of popular data types, such as space partitioning trees, as well as a family of digital trees like suffix tree, patricia. We present prototype of new indexing infrastucture, called SP-GiST, which adds support of these data types to PostgreSQL. Known implementation of SP-GiST based on academic research paper can't be used in production environment, since it's compatible with rather old version of PostgreSQL and doesn't supports concurrency and recovery, so we propose to develop built-in SP-GiST infrastructure and would like to discuss several implementation issues and development roadmap. Oleg Bartunov Teodor Sigaev 13:30 01:00 DMS 1160 Hacking lecture en What does the planner do and why does it do it? How can it be improved? PART II This is a continuation of Tom's talk from yesterday. This is a follow-on to Robert Haas's planner talk at PGCon 2010. I plan to focus more on why and how the planner works, ie, present a developer's-eye view instead of a user's-eye view of what it does. Topics will include the overall layout of the planner code, preprocessing, join search order, and cost estimation. Hopefully attendees will come away encouraged to hack on the planner and help improve it. Tom Lane 15:00 01:00 DMS 1160 Optimal PGXN exposure via Smart distribution packaging 9.1 Features lecture en So you've solved a common problem in PostgreSQL, a serious pain point that lots of other people could take advantage of. You want to open-source the code, and would like to package it up and distribute it where people can find it, read its docs, and download and install it. The newest destination for distributing PostgreSQL extensions is [PGXN](http://pgxn.org/), the PostgreSQL Extension network. PGXN offers a central index of PostgreSQL extensions and their documentation, making it easy for people to find an install extensions. How do you optimize your extension distribution for maximum PGXN exposure? By careful packaging, thorough documentation, and simple installation. Join me for an overview of how best to achieve that — including how to build extensions compatible with PostgreSQL 9.1's improved extension support. One of the primary distinguishing features of PostgreSQL—and perhaps the number one reason to use it instead of another DBMS—is its extensibility and the large number of database extensions already available. PostgreSQL today is not merely a database, it’s an application development platform. So you've solved a common problem in PostgreSQL, a serious pain point that lots of other people could take advantage of. So you've decided to open-source the code, and would like to package it up and distribute it where people can find it, read its docs, and download and install it. The newest destination for distributing PostgreSQL extensions is [PGXN](http://pgxn.org/), the PostgreSQL Extension network. PGXN offers a central index of PostgreSQL extensions and their documentation, making it easy for people to find an install extensions. How do you optimize your extension distribution for maximum PGXN exposure? By careful packaging, thorough documentation, and simple installation. Join me for an overview of how best to achieve that — including how to build extensions compatible with PostgreSQL 9.1's improved extension support. David E. Wheeler PGXN PGXN Manager 16:30 01:00 DMS 1160 both of the feature and developments using it 9.1 Features lecture en PostgreSQL extensibility is remarkable but incomplete. It lacks dump and restore support. What that means is that once an extension is installed into your database, PostgreSQL currently has no idea of what SQL objects belongs to the extension rather itself, so the dump will contain the instructions to install the extension. That's only practical if you want to restore your dump targeting the very same extension's version, but when upgrading systems that's seldom what happens. This talk will detail how to fix this problem and more, explaining you how to benefit from the extensions capabilities for your own work within the database. Dimitri Fontaine 17:30 01:00 DMS 1160 prizes, auctions, fun, games Plenary other en The Traditional Closing Session Watch the video. We raised thousands for charity. Dan Langille audio 10:00 01:00 DMS 1120 9.1 Features lecture en PostgreSQL v9.1 shall be released with a long-awaited feature that enables to apply label based mandatory access control. It was primarily developed to integrate security model of SELinux, however, it was evolved to more generic design by community efforts. Nowadays, we can apply an additional security mechanism from a difference perspective in addition to the default database privileges. This session offers a brief overview of this model, an introduction of what's new in v9.1 and limitations to be overcome in the future release. This session will be consist of three parts. The first one is a brief overview of label based mandatory access control; using SELinux as an example. A key concept is reference monitor model and criteria to make access control decision across heterogeneous object managers; such as filesystem and RDBMS. The second one is what's new in v9.1. This version newly provides a basic facilities to manage security labels and security hooks on accesses. It enables us to implement external security provider as a plug-in module; such as SELinux support. The last one is limitations of the current version and future works. However, we are still mid-way towards the fully expected functionalities, so we still have several limitations. We will also introduce our roadmap and elemental technologies being necessary to overcome. KaiGai Kohei 11:30 01:00 DMS 1120 A Revolution in SQL 9.1 Features lecture en In SQL, Common Table Expressions are a way of creating and using views at run time, but only in read queries. Here we discover ways to use them in writes. Find out about this exciting new feature! We can make Common Table Expressions even more powerful than the current SQL standard allows. We might even make it part of a future SQL standard. Old syntax: WITH [RECURSIVE] cte1 [(col[, col...])] AS ( READ_QUERY [(UNION|INTERSECT) [ALL] READ_QUERY )[, cte2 ...] SELECT ... New syntax: WITH [RECURSIVE] cte1 [(col[, col...])] AS ( ROW_RETURNING_QUERY /* for example, DELETE...RETURNING */ [(UNION|INTERSECT) [ALL] ROW_RETURNING QUERY )[, cte2 ...] (SELECT|INSERT|UPDATE|DELETE) ... David Fetter 13:30 01:00 DMS 1120 An discussion of HTSQL's join algebra and translation to SQL Advanced Features lecture en This is advanced talk will be an in-depth discussion of HTSQL's query language's design, focusing primarily on techniques used for join specification and translation into SQL. It should be of broad interest to core PostgreSQL core developers. It will likely not be that useful for application developers or users of HTSQL since the talk will be focusing on translation mechanics. In the notes at the end of the SQL 2003 specification, there is a suggested link feature (-> syntax) which could be used to relate tables via declared foreign keys. It's been 8 years and this sort of feature still doesn't exist in the core of modern relational databases. It's not because the idea is bad, it's more likely that it's because implementing it is non-trivial. HTSQL has implemented this sort of mechanism in a manner that is formally defined. This talk will provide an overview of what we've done to get something like this working, the sort of internal bookkeeping needed and lessons learned. The talk will be given by Clark Evans & Kirill Simonov. HTSQL is an high-level URI-based relational database query language designed for accidental programmers. It is particularly good at complex reporting and interactive dashboard construction. HTSQL was continuously refined over the last 7 years with user feedback and has become a very useful tool for not only casual database users, but developers wishing to add ad-hoc query capability to their application. Clark C. Evans HTSQL Website Slides for 5/20 http:// 15:00 01:00 DMS 1120 How PostgreSQL is being used at www.worldflightplanner.com GIS lecture en Postgres is a powerful database, especially when coupling it with PostGIS to make it spatially aware. Using PostgreSQL and PostGIS allows World Flight Planner to provide advanced flight planning tools to both private and commercial pilots. Using PostgreSQL and PostGIS allows you to access and modify your spatial data using SQL. This talk will provide an overview on the advantages to using PostGIS, how to properly store and query your geographic data, and offer some solutions to common problems. This talk will comprise: - A primer on storing geographical data in postgres - Everything you needed to know about projections, sphereoids. - Cartesian Trig Vs. Spherical Trig - Types of data we need to store (points, line strings, other geometries) - Why we decided to use Postgres+PostGIS - How we store global scale meteorological data (converting gridded data in binary format into rows in the postgres database for fast querying) - Some of the challenges for storing aeronautical information in postgres. - How the data was organized in the database (a huge challenge, as identifiers for airports and navigational aids are not globally unique) - Odd airspace shapes (such as pie, or "pac man" shapes) - Describe the overall relationships of aeronautical data (airports, navigational aids, fixes) with each other. - Some custom functions we wrote to get the job done. - Examples of how to use PostGIS functions to do fast computations, such as: - Find me the closest airport to my current location. - Does my flight path take me through restricted airspace? - What heading do I need to take when I cross my next waypoint (bearings). - Displaying geospatial data, How we did it (map server, mapnik, openlayers, etc..) Blake Crosby 16:30 01:00 DMS 1120 Making ISOLATION LEVEL SERIALIZABLE Provide Serializable Isolation 9.1 Features lecture en Serializable isolation makes it easier to reason about database behavior because concurrent transactions appear to have executed sequentially. However, Postgres has historically only provided snapshot isolation, even at the SERIALIZABLE isolation level, which can expose anomalous states to queries. This talk will describe a new SERIALIZABLE isolation level using Serializable Snapshot Isolation (SSI). Unlike the typical two-phase locking approach to serializability, SSI uses the existing snapshot isolation mechanisms and retains most of their performance benefits. It adds a new type of lock that's used to track read dependencies and roll back transactions when dangerous structures appear in the serialization graph, indicating possible anomalies. We'll describe how SSI works and how we've implemented it for Postgres 9.1, including a new predicate lock manager. We'll explain what users can expect when using the new SERIALIZABLE mode, and present some performance results. Dan Ports Kevin Grittner 09:30 04:00 DMS 1110 pldevs PL/Mania Community meeting en Procedural Language Eevelopers will met and discuss current issues around PLs Selena Deckelmann 10:00 05:00 Out and about tourist Spend some time exploring Social other en Explore Ottawa Ottawa has a large number of great attractions. Spend some time looking around and explore. Spend as much time as you want with us, or leave early. We will walk everywhere we go. Wear sensible shoes. Bring your camera. We'll probably have lunch somewhere along the way. Consider the weather (sun block, rain coat, umbrella, swim suit). This is one option for Saturday. You are free to come with us or take part in all or some activities. Or make your own way around... The agenda for the tourist day is completely wrong. The forecast: http://www.theweathernetwork.com/weather/CAON0512 Cora's - 8 AM - Breakfast - 179 Rideau Street - http://bit.ly/dngpnV - if you arrive much past 8:30, you probably won't get through in time to get to the next meeting point - we will depart Cora's at 8:55 Rideau Center - 9:13 - south end of the Mall on the Mackenzie King bridge - we are catching the #95 bus to direction Orleans to BLAIR 1B - from there, at 9:27, take Bus route 129 (OC Transpo) direction Aviation Aviation Museum - we'll do a guided tour first, then wander around - departing here at 1:30 Earl of Sussex Pub - arriving at about 2:15 - 431 Sussex Drive Ottawa, ON K1N 9M6 (613) 562-5544 earlofsussex.ca All times after we leave Rideau Center are very subject to change. Dan Langille National Memorial Residence Forum