PGCon 2011The PostgreSQL ConferenceUniversity of OttawaOttawa2011-05-172011-05-215Add 4 Video (2015.09.18)09:0000:3009:0003:00DMS 1120Using The PostgreSQL System CatalogsDatabase Introspection with PostgreSQLworkshopenThe PostgreSQL system catalogs store metadata about SQL objects such as tables, columns, functions, schemas, and more. This talk will explore the organization and structure of the system catalogs and explain how you can use these catalogs to manage and tune your database.We'll cover applications such as (1) getting a list of all tables that have no indices, (2) determining table and database sizes on disk, and (3) monitoring table, session, and background writer activity, as well as the internal structure of the catalogs and how they're used by the database system.Robert Haas
video
13:0003:00DMS 1120PostgreSQL Performance PitfallsKeeping yourself out of the pitworkshopenPostgreSQL is a database system that can deliver excellent performance for a wide variety of applications. But it's easy to run into an issue that keeps you from seeing its full potential. There are a few basic PostgreSQL configuration and use misunderstandings that cause most of the early performance issues administrators and developers encounter.This talk covers the most common performance problems seen in real-world PostgreSQL deployment. It briefly summarizes each and points out the basics of how to avoid them. It's like navigating a minefield: you'll progress though a lot faster if you can start by knowing what the mines look like.
With PostgreSQL 9.0 featuring all new built-in replication and query distribution features, the timing has never been better to consider what a serious jump into using PostgreSQL would look like. Getting the performance related aspects right can be one of the most stressful parts of such a migration, particularly if you're new to the database and its quirks. Follow the suggestions here, and you can build your database to postpone many of these slowdowns until you've already proven your application is a success.Greg Smith
video
09:0003:00DMS 1120Life with Object-Relational MappersOr, how I stopped worrying and learned to love the ORMworkshopenObject-Relational Mappers (ORMs) are the primary interface that application programmers now use to interact with SQL databases. Any DBA can list the problems that they create, but how can we move on to solutions that don't involve a complete reengineering the ORM-database interaction paradigm? Rather than simply declaim that ORMs are bad, let's find a way of mitigating the problems that existing ORMs create.The database community seems to have been blindsided by the rise of Object-Relational Mappers (ORMs) as the primary database interface technology, even though ORMs themselves have been around for years (the core concepts date to the early 1990s and before). The problems with ORMs are real: Poor-quality SQL, schemas that are not well-matched to the capabilities of the database, and database access patterns that are hugely inefficient.
Yet ORMs are here to stay, and we must deal with ORMs that currently exist, rather than the theoretical ORMs we wish we had. As database architects and engineers, ORMs are now part of our client universe, and we need to understand them and find ways of working with, rather than against them.
We'll talk about why ORMs have the characteristics that they do, and why they appeal so strongly to application programmers over direct-SQL model. Using real-life ORMs, we'll explore ways that database architects can guide the development of ORM-based applications in a way that mitigate the problems that an ORM can create in database interaction. These include ORM-friendly changes to the application, ORM-invisible changes to the database, and (when required) spot alternations to the application to avoid disasters.
Christophe Pettus
video
13:0003:00DMS 1120Trees and Graphs in the databaseRDBMS in the social networks ageworkshopenDespite the NoSQL movement trying to flag traditional databases as a dying breed, the RDBMS keeps evolving and adding new powerful weapons to its arsenal. In this talk we'll explore Common Table Expressions (SQL-99) and how SQL handles recursion, breaking the bi-dimensional barriers and paving the way to more complex data structures like trees and graphs, and how we can replicate features from social networks and recommendation systems. We'll also have a look at window functions (SQL:2003) and the advanced reporting features they make finally possible.The first part of this talk will cover several different techniques to model a tree data structure into a relational database: parent-child (adjacency list) model, materialized path, nested sets, nested intervals, hybrid models, Common Table Expressions.
Then we'll move one step forward and see how we can model a more complex data structure, i.e. a graph, with concrete examples from today's websites. Starting from real-world examples of social networks' and recommendation systems' features, and with the help of some graph theory, this talk will explain how to represent and traverse a graph in the database.
Finally, we will take a look at Window Functions and how they can be useful for data analytics and simple inline aggregations, among other things.
All the examples have been tested on PostgreSQL >= 8.4Lorenzo Alberton
Trees in the database - Slides
Graphs in the database - Slides
15:0007:00Royal OakregistrationRegistration pickupThe social way to register: at the pubotherenPick up your registration packStop by the Royal Oak Pub and get your registration pack. You'll help us avoid long line ups on Friday morning and you get to have a beer, and chat with your fellow attendees. We guarantee you'll spot someone famous.Dan Langille10:0001:00DMS 1140Database Scalability Patterns: Sharding for Unlimited GrowthMost people have come to understand simple replication based horizontal scale-out, but while this technique works pretty well for some read loads, there are system where either the amount of data involved is too great for a single system, or the write based load goes beyond the capacity of a single machine. In these cases we must turn to complex systems of federated databases to build database systems that can sustain perpetual growth. "Sharding" is a buzz word losing some of it's buzz, and that's a good thing. For many people building read-based scaleout systems is all they will ever need, but if it isn't where do you turn? In this talk I'll focus specifically on large scale, application based scale out being used by some of todays largest internet infrastructures. We'll focus on understanding the pattern underlying these systems, how they are implemented (including discussions on MySQL vs. Postgres as a backing store), and how to manage the infrastructure once you've built it. It isn't for the faint of heart, but if you plan to build large scale systems on commodity hardware, you need to know what has worked for others in the past. Robert Treat
video
11:3001:00DMS 1140Review of Patch ReviewingHow to Review Patches for PostgreSQLlecturePatch reviewing is an important part of moving PostgreSQL forward. It's critical for anyone submitting patches to PostgreSQL to also understand how to review patches, too. This talk will go through the steps of patch review, including the most important piece- code review.PostgreSQL, more than most applications, must be stable and reliable. Crashes are just not acceptable, ever. Data loss must be prevented. Add to this the many complexities of meeting the requirements of ACID and SQL, and it's clear that the code in PostgreSQL needs to be reviewed very carefully. We depend on the committers to do final review and commit, but they are a very finite resource. The PostgreSQL community has an excellent approach to scaling up our committer resources- make sure what they're reviewing is of good quality.
We'll talk about how to become a reviewer, how to use the CommitFest application, what makes a good pre-committer review, some tricks to doing reviews, and what to look for when doing a code review.Stephen Frost13:3001:00DMS 1140Real Federation Database System leveraging PostgreSQL FDWlectureenFederation Database System solves transparent access to heterogeneous DBMS to realize Optimized Database schema design according to optimized business process. Our database team develops Real Federation Database System leveraging PostgreSQL FDW. This federation system enhances the current optimization of Foreign Scan and adds Foreign Data Wrapper for another RDBMS such as Oracle and Microsoft SQL Server.We implement Federation Database System using PostgreSQL FDW and realize following things:
- Access heterogeneous external data source as local data
- Location, Vendor and Platform Transparency
- Without change of existing system change
- Real-time Data Reference and Data Update
These features give us various advantages as follows:
- Optimization of Business Process across multiple business systems
- Integrated Database access
- Avoidance of redundant data management
My presentation explains the development technique for Federation Database System based on PostgreSQL FDW. It includes the optimization technique for heterogeneous distributed database access and integration technique of Foreign Data Wrapper to heterogeneous RDBMS (e.g. Oracle and Microsoft SQL Server) and so on. And also we would like to discuss the requirements for new Federation Database System for this cloud era.Yotaro Nakayama15:0001:00DMS 1140Understanding PostgreSQL LWLockslectureenIn this session we take a look at the Light Weight Lock facility in PostgreSQL and the various lwlocks.PostgreSQL uses Light Weight Locks to manage PostgreSQL Locks.
In this session we take
* Describe the framework of PostgreSQL LWLocks facility
-> Architecture of the LWLocks Framework
-> Lifecycle of an LWLock using its API
-> Understanding Exclusive and Shared modes of lwlock
-> Internals of LWLock Wait List
-> Ways to monitors the events related to LWLocks
* Top LWLocks.
-> WALInsertLock
-> WALWriteLock
-> ProcArrayLock
-> SInvalReadLock
-> CLogControlLock
-> DynamicLWLocks
Once you start monitoring LWLocks, you will probably find one of the above locks as highly used or contended. We go a bit further in understanding what this lock means and the corrective actions to take to reduce the stress on the locks.
Jignesh K. Shah
Presentation Slides
16:3001:00DMS 1140PostgreSQL LoggingHow to get the most out of your activity loglectureenLogs can tell you a lot about what's going on in your database: which user logged in when, what queries they ran and how long they took, when the database restarted. We'll walk through the various logging configuration options PostgreSQL offers to help you figure out how to get what you need in a format that's useful to you. We'll also talk about what to do with your logs once you have them (hint: you don't have to stay up at night reading through them!) and, time permitting, wrap up with a group discussion.Logging is an important part of your database monitoring suite, providing historical information for troubleshooting and forensic purposes: which user logged in when, what queries they ran and how long they took, when the database restarted. PostgreSQL offers different file format, message format, and message content options for logging. These options can be kind of overwhelming for the first-time user. We'll walk through the various configuration parameters to help you figure out how to get what you need in a format that's useful to you. Then we'll discuss some solutions (pgFouine and friends) for extracting information from your logs. If there's time, I really like to have a group discussion at the end of my talks - so bring your configs, logs, and war stories!Gabrielle Roth09:0001:00DMS 1160keynotePostGIS Knows Where You AreShould you be Worried?lectureenPaul Ramsey has been working with geospatial software for more than 10
years, as programmer and consultant. He co-founded the PostGIS spatial
database project in 2001, and is currently an active developer and
member of the project steering committee. In 2008, Paul received the
Sol Katz Award for achievement in open source geospatial software.
Geolocation is a hot topic in technology right now,
so this topic will offer a tour of the terrain: what geolocation is
being used for, where PostGIS came from, how it uses PostgreSQL, who
is using it, and the technology challenges that PostGIS (and by
extension PostgreSQL) face in the future as use cases continue to push
more and more data through the system.
Paul Ramsey 10:0001:00DMS 11609.1 Mystery TourWhat's in the upcoming versionSQL/MED, Sync Rep, SSI, Streaming Backups, SE-PostgreSQL, Writeable CTEs, Range Types, per-Column Collations, EXTENSIONs, directory archives, checkpoint goggles, VIEW triggers, extensible ENUMs, and more.What are these things? Why should they matter to you? Which of them are in 9.1 and which aren't?Josh Berkus11:3001:00DMS 1160Best practices with Ora2PgMigrating Oracle databases to PostgresQLlectureenOra2Pg is a GPLv3 licensed tool used to help migrate an Oracle database to a PostgreSQL compatible database. Ora2Pg can be used from reverse engineering Oracle database to huge enterprise database migration or simply to replicate some Oracle data into a PostgreSQL database.
This talk by the author of Ora2Pg will start with an introduction to the project and its history. We will continue with the best practices using Ora2Pg for Oracle to PostgreSQL migration. We will cover much of the functionalities coming in the v7.0 release which contains significant speed improvement and new features. An overview of the future development of Ora2Pg will be provided.
The talk will also cover the basic techniques and gotchas around migrating from Oracle to PostgreSQL.Gurjeet Singh
Home page of the project
13:3001:00DMS 1160bigpostgresMaintaining Terabytes10 Things to Watch Out For When PostgresSQL Gets BiglectureenSize can creep up on you. Some day you may wake up to a multi-terabyte Postgres system handling over 3000 tps staring you down. Learn the best ways to manage these systems as they grow, and find out what new features in 9.0 have made life easier for administrators and application developers working with big data.Size can creep up on you. Some day you may wake up to a multi-terabyte Postgres system handling over 3000 tps staring you down. Learn the best ways to manage these systems as they grow, and find out what new features in 9.0 have made life easier for administrators and application developers working with big data.
This talk will lead you through solutions to problems Postgres faces when it gets big: backups, transaction wraparound, bloat, huge catalogs and upgrades. You need to monitor the right things, find the gems in DBA-friendly database functions and catalog tables, and know the right places to look to spot problems early. We’ll also go over monitoring best practices and open source tools to get the job done.
Working with multiple versions of Postgres back to version 8.2 will be included, and as well as tips on making the most out of new features in 9.0. War stories will be taken from real-world work with Emma, an email marketing company with a few large databases.Selena Deckelmann15:0001:00DMS 1160Parallel Image Searching Using PostgreSQL and PgOpenCLRunning PostgreSQL Stored Procedures on a GPUlectureen2D and 3D Image data is increasing important. New 3D scanning devices are being used to map a variety of large and small spaces and objects.
Large GIS data sets are being acquired with new scanners. Computation photography allows the manipulation of images for many applications both scientific and social. GPUs are increasingly being used to accelerate image based computations. This presentation illustrates how PostgreSQL can store and process images using GPU hosted stored procedures, written in OpenCL.This is an outline of my proposed slides
• Title
• Speaker’s Bio
o Brief biographical details of the speaker.
• Talk Outline
o Overview the talks topics
• Parallel Universe
o Provides an over view of what GPU (highly parallel chips) will bring to the DBMS world.
• The power of GPU’s
o Describes the enormous computing power available from GPUs.
• OpenCL GPU/CPU Programming Language
o Describes the new OpenCL programming language that can be used for parallel programming for CPUs and GPUs.
• PgOpenCL a new PostgeSQL Procedural Language
o Describes PgOpenCL a new PostgreSQL procedure language that allows stored procedures to be written in OpenCL.
• Image Data Types
o Outlines 2D,and 3D image data types that are used to storage and processing of images.
• Parallel Image searching
o Outlines the type of image filter functions that can be used to process images and extract information as SQL tables from images.
• Future Directions
o Describes the future direction of parallel processing in databases.
• Summary
o Summarized the main points of the talk
• Q &A
o Time to answer question from the audience.
Tim Child
PgCon 2011 Slides
16:3001:00DMS 1160Securing PostgreSQL From External AttacklectureenThis talk explores the ways attackers with no authorized database access
can steal Postgres passwords, see database queries and results, and even
intercept database sessions and return false data. Postgres supports
features to eliminate all of these threats, but administrators must
understand the attack vulnerabilities to protect against them. This talk
covers all known Postgres external attack methods.
Bruce Momjian17:3001:00DMS 1160lightningLightning talksShort sharp descriptions of short topicslightningenA regular feature, PGCon will have a Lightning talks session, with presentations on diverse topics.The format remains essentially the same: in a one hour period, audiences are entertained and informed by a rapid fire series of short talks on interesting new or on-going work by individuals or groups. Slides aer permitted, but not obligatory; pictures are highly recommended. Topic areas include new open source software projects, works in progress for future releases of existing projects, student projects, etc. Lightning talks topics this year may make good conference papers next year!
The number of slots is limited, and experience suggests there will be more takers than slots. Sign up well in advance to be assured a spot. The session chair this year is Selena Deckelmann.
Magnus HaganderSelena Deckelmann
List of talks + slides
10:0001:00DMS 1120Data-driven cache invalidation lectureenIn todays world, nobody (should) deploy a web application facing the
internet without having a proper caching system in place. There are
many different solutions to choose from, from manual use of memcached
through framework integrated caching to external caches like Squid or
Varnish. Most modern frameworks come with integrated functionality for
at least one of these methods, and often more than one.However, they often relies on all traffic going through the same
framework to work properly - a caching layer in Rails is hard to share
with one in Django. This talk will show a way to break the design
rules of these frameworks just a little, and have the database help
solve this problem.
This talk will use a small application written in Python using Django
to illustrate the examples, but the method is language independent.
Unsurprisingly, the database used is PostgreSQL.Magnus Hagander11:3002:00DMS 1120MVCC UnmaskedlectureenMulti-Version Concurrency Control (MVCC) allows Postgres to offer high
concurrency even during significant database read/write activity. MVCC
specifically offers behavior where "readers never block writers, and
writers never block readers".This talk explains how MVCC is implemented
in Postgres and highlights optimizations which minimize the downsides of
MVCC. This talk is for advanced users.
This is a two hour talk which extends into lunch. There will be a short break to allow people to grab lunch and bring it back into the lecture hall.
Bruce Momjian13:3001:00DMS 1120How To Get Your PostgreSQL Patch AcceptedIn short: -1 from me. regards, tom lanelectureenWriting a PostgreSQL patch can be difficult, and getting it accepted can require even more work than writing it in the first place.Many people contribute to PostgreSQL each year, and these contributions are critical to PostgreSQL's ongoing success and continued improvement. Patches are also regularly turned away for a variety of reasons. In this talk, I'll discuss how to maximize your chances of a successful contribution and what to do next if your patch is rejected.Robert Haas15:0001:00DMS 1120NTT’s Case ReportIntroduce PostgreSQL into reliable and large-scale telecommunication support systemlectureenWe will present how to introduce PostgreSQL into NTT’s telecommunication systems. To inform the merit of PostgreSQL for IT system managers we evaluated its performance and showed applicable field of our production systems. To broaden the field we have resolved performance problems like as performance glitch at checkpoints and improved its usability with home-made tools such as pg_bulkload, pg_statsinfo and so on. All these efforts and their results are worth being shared with who plan and proceed PostgreSQL’s enterprise use.NTT group, which is the largest telecommunication career in Japan providing more than 120 million subscribers, has made effort introducing PostgreSQL to its production systems that support telecommunication. Such systems require as same performance and reliability as proprietary DBMSes provide, which shows PostgreSQL should be enhanced in many aspects for enterprise use.
We will present what kind of performance characters were required from production systems engineers and how we showed them. At first we evaluate PostgreSQL performance under TPC-C and TPC-W like workload to show its throughputs are enough for many systems. At the same time we found instability of response time and complex maintenance operation like VACUUM, which we thought be large obstacles for introduction. We tried to resolve them and they are finally cleared up by PostgreSQL 8.3. From the view point of usability, our production engineers pointed PostgreSQL lacked some essential utility tool such as fast data-loader and performance monitor. So we provided original open-source tools pg_bulkload and pg_statsinfo to meet such requirements.
Finally we will present improvement availability by PostgreSQL 9.0 and show expected applied systems that the feature should enable.
Tetsuo Sakata16:3001:00DMS 1120Hacking the Query PlannerlectureenWhat does the planner do and why does it do it? How can it be improved?This is a follow-on to Robert Haas's planner talk at PGCon 2010. I plan to focus more on why and how the planner works, ie, present a developer's-eye view instead of a user's-eye view of what it does. Topics will include the overall layout of the planner code, preprocessing, join search order, and cost estimation. Hopefully attendees will come away encouraged to hack on the planner and help improve it.Tom Lane18:3002:00The Velvet RoomsocialoutingMajor Social Event!sponsored by EnterpriseDBotherenCome and join us for the major evening social event of PGCon 2011!EnterpriseDB is sponsoring this event for all PGCon 2011 attendees. Dinner and drinks will be provided. See the map on the website for directions to the venue.
The Velvet Room
62 York St
Ottawa, ON K1N 8Y3
6:30pm-8:30pmDan Langille
Map
10:0001:00DMS 1140Android & PostgreSQLHow to develop an Android App for PostgreSQLworkshopenThis is a presentation on how-to develop a simple Android application that uses the PostgreSQL JDBC driver to communicate directly with a database. Use the Android SDK with Eclipse to build and deploy an application. Familiarity with Java programming and XML recommended.Learn how to use the Android SDK and Eclipse to develop a simple user interface to connect to a PostgreSQL database and execute simple SQL statements. Programming tasks include:
* Setting up a development environment
* Executing simple SQL statements
* Executing SQL statements with JDBC Prepared Statements and bind variables
* Using cursors to handle potentially large volumes of data
* Packaging and distributing Android applicationsMark Wong11:3001:00DMS 1140Advanced WAL file management with OmniPITR. lectureenOmniPITR is a set of scripts designed to help manage and maintain WAL files for Postgres database. It was created in an effort to replace the myriad scripts we've encountered, and written, managing our clients Postgres installs over the years. It includes tools for both archiving, restoration, and creating portable backups on both the master and slave instance. Ever since the advent of PITR in Postgres 8.1, people have been creating 1 off scripts designed to manage PITR and the xlog files that come with it. We've been just as guilty of this as anyone else, making specialized script after script for clients, tweaking for one difference or another. Finally last year we decided we could do better, and OmniPITR was born.
OmniPITR is a set of scripts designed to help manage and maintain WAL files for Postgres database. It was created in an effort to replace the myriad scripts we've encountered, and written, managing our clients Postgres installs over the years. It includes tools for both archiving, restoration, and creating portable backups on both the master and slave instance.
Designed to work with Postgres 8.2 and above, we've now replaced a whole mish-mash of scripts with something far more standard and re-usable. Cross platform and Postgres licensed, we have started to see more and more adoption outside of our client base, so we feel comfortable that it's a stable option for many of todays most demanding environments.
Robert Treat
OmniPITR source
13:3001:00DMS 1140Doing Bioinformatics in PostgreSQLlectureenWe introduce and describe two modules that grew from the need to perform
integrated and efficient Bioinformatics tasks in PostgreSQL: PostBio, a set of
methods to store and query genomic sequences and features, and PostStat, a
collection of statistical functions that allow for integrated statistical
tests. A few practical examples will be presented to showcase the modules.PostBio includes three data types: a GiST-indexable integer interval used to
represent biological sequence features; a suffix tree type to search for
maximum unique matches; and a compressed suffix array for fast short exact
matches. In addition, PostBio provides a set of utilitary sequence routines.
PostStat comprises routines that compute a number of cumulative probability
distributions, linear regression, and statistical tests, both parametric and
non-parametric; the main motivation is to provide a way to test statistical
hypothesis in simple models.Luis Carvalho15:0001:00DMS 1140The Write StuffReaching peak write performance under heavy loadslectureenOne of the most difficult things to do with a database system is handle a high volume of writes, whether it's from a single user or thousands. The steps a database like PostgreSQL takes to protect your data are expensive, and it's hard to split them into independent pieces. Recent changes available in the upcoming PostgreSQL 9.1 have given new insight into exactly how PostgreSQL systems that write constantly break down under serious loads.The replication features in PostgreSQL 9.0 make it easier than ever to scale your application performance up by adding more read-only nodes with additional copies of the data. But that also makes it easier than ever before to end up running into a harder to resolve write bottleneck as a result.
This talk will cover the major approaches and trade-offs in tuning for high write performance on a PostgreSQL server. The write-ahead log, checkpoint timing, and buffer management policies of the database all interact to limit just fast data can be shuffled in and out of the server. Some recently added features to PostgreSQL 9.1 will also be introduced. These give better insight than was ever available before into exactly what caused the kind of checkpoint write "spikes" that have traditionally plagued write-heavy PostgreSQL databases. The trade-offs between optimizing for latency and throughput are highlighted when watching operations at this level.
Understanding these problems can help you make the right long-term choices about how to tune system, or provide ammunition for why you need to plan an upgrade to 9.1 when it becomes available in the future.Greg Smith16:3001:00DMS 1140Running PostgreSQL in enterprise level telco systemslectureenA specific example of using PostgreSQL in enterprise level OSS system for largest pan-russian cellular provider - technology choice reasoning, implementation, maintenance and performance issues.Solution overview and implementation history. Pessimistic and critical comparison PostgreSQL vs commercial database servers, reasons for choosing PostgreSQL. Expected and unexpected problems using Postgres in the field of telecommunications. Solutions and review of still unsolved difficulties. Postgres specific benefits. All these issues are described in the context of database administration and software development, as well as of interaction with the customer and the vendor.Ilya Kosmodemiansky10:0001:00DMS 1160Range TypesRange Types and Temporal DatalectureenRange Types are a new feature that captures ranges -- ranges of time, ranges of numbers, ranges of dates -- and gives users the ability to create their own range types. Using Range Types improves performance by using range indexes, simplifies your queries and schema, and enables advanced features such as using Exclusion Constraints to prevent overlap.Most organizations need to handle range types in one way or another. Ranges of numbers or IP addresses are useful; and of course, ranges of time are perhaps the most common. Explicitly telling PostgreSQL that you are working with ranges -- rather than simulating ranges with a "start" and "end" column -- offers major benefits for functionality (e.g. the ability to prevent overlap), performance (GiST indexing), and ease of use.
We'll walk through some basic examples, compare with the "two-column approach" (that is, using a "beginning" and an "end" column), show the benefits of range types using temporal data as an example, and give a brief overview of how to define your own range types.Jeff Davis11:3001:00DMS 1160SP-GiST - a new indexing infrastructure for PostgreSQLSpace-Partitioning trees in PostgreSQLlectureenWe want to present a prototype of new indexing infrastructure, called SP-GiST, which greatly extends current extendability of PostgreSQL to support space-partitioning trees, as well as a family of digital trees like suffix tree, patricia. We'll present programming interface, examples of realizations of some useful datatypes. Also, we'd like to discuss some implementation issues and roadmap for inclusion to PostgreSQL.PostgreSQL is known as an extendable database, which means that other than built-in data types can be developed by experts in specific domain, not core programmers. Currently, there are GiST (Generalized Search Tree) and GIN (Generalized Inverted Index) infrastructure in PostgreSQL, which provides extendability with indexing, concurrency and recovery support for user-defined data types. However, these infrastructures are not enough to support a wide class of popular data types, such as space partitioning trees, as well as a family of digital trees like suffix tree, patricia. We present prototype of new indexing infrastucture, called SP-GiST, which adds support of these data types to PostgreSQL. Known implementation of SP-GiST based on academic research paper can't be used in production environment, since it's compatible with rather old version of PostgreSQL and doesn't supports concurrency and recovery, so we propose to develop built-in SP-GiST infrastructure and would like to discuss several implementation issues and development roadmap.Oleg BartunovTeodor Sigaev13:3001:00DMS 1160Hacking the Query Planner IIlectureenWhat does the planner do and why does it do it? How can it be improved? PART II
This is a continuation of Tom's talk from yesterday.This is a follow-on to Robert Haas's planner talk at PGCon 2010. I plan to focus more on why and how the planner works, ie, present a developer's-eye view instead of a user's-eye view of what it does. Topics will include the overall layout of the planner code, preprocessing, join search order, and cost estimation. Hopefully attendees will come away encouraged to hack on the planner and help improve it.Tom Lane15:0001:00DMS 1160Distributing Extensions on PGXNOptimal PGXN exposure via Smart distribution packaginglectureenSo you've solved a common problem in PostgreSQL, a serious pain point that lots of other people could take advantage of. You want to open-source the code, and would like to package it up and distribute it where people can find it, read its docs, and download and install it.
The newest destination for distributing PostgreSQL extensions is [PGXN](http://pgxn.org/), the PostgreSQL Extension network. PGXN offers a central index of PostgreSQL extensions and their documentation, making it easy for people to find an install extensions.
How do you optimize your extension distribution for maximum PGXN exposure? By careful packaging, thorough documentation, and simple installation. Join me for an overview of how best to achieve that — including how to build extensions compatible with PostgreSQL 9.1's improved extension support.One of the primary distinguishing features of PostgreSQL—and perhaps the number one reason to use it instead of another DBMS—is its extensibility and the large number of database extensions already available. PostgreSQL today is not merely a database, it’s an application development platform.
So you've solved a common problem in PostgreSQL, a serious pain point that lots of other people could take advantage of. So you've decided to open-source the code, and would like to package it up and distribute it where people can find it, read its docs, and download and install it.
The newest destination for distributing PostgreSQL extensions is [PGXN](http://pgxn.org/), the PostgreSQL Extension network. PGXN offers a central index of PostgreSQL extensions and their documentation, making it easy for people to find an install extensions.
How do you optimize your extension distribution for maximum PGXN exposure? By careful packaging, thorough documentation, and simple installation. Join me for an overview of how best to achieve that — including how to build extensions compatible with PostgreSQL 9.1's improved extension support.David E. Wheeler
PGXN
PGXN Manager
16:3001:00DMS 1160Extensions Developmentboth of the feature and developments using itlectureenPostgreSQL extensibility is remarkable but incomplete. It lacks dump and
restore support. What that means is that once an extension is installed
into your database, PostgreSQL currently has no idea of what SQL objects
belongs to the extension rather itself, so the dump will contain the
instructions to install the extension. That's only practical if you want
to restore your dump targeting the very same extension's version, but
when upgrading systems that's seldom what happens. This talk will detail
how to fix this problem and more, explaining you how to benefit from the
extensions capabilities for your own work within the database.
Dimitri Fontaine17:3001:00DMS 1160Closing sessionsprizes, auctions, fun, gamesotherenThe Traditional Closing SessionWatch the video. We raised thousands for charity.Dan Langille
audio
10:0001:00DMS 1120Label Based Mandatory Access Control on PostgreSQLlectureenPostgreSQL v9.1 shall be released with a long-awaited feature that enables to apply label based mandatory access control. It was primarily developed to integrate security model of SELinux, however, it was evolved to more generic design by community efforts. Nowadays, we can apply an additional security mechanism from a difference perspective in addition to the default database privileges.
This session offers a brief overview of this model, an introduction of what's new in v9.1 and limitations to be overcome in the future release.This session will be consist of three parts.
The first one is a brief overview of label based mandatory access control; using SELinux as an example.
A key concept is reference monitor model and criteria to make access control decision across heterogeneous object managers; such as filesystem and RDBMS.
The second one is what's new in v9.1.
This version newly provides a basic facilities to manage security labels and security hooks on accesses. It enables us to implement external security provider as a plug-in module; such as SELinux support.
The last one is limitations of the current version and future works.
However, we are still mid-way towards the fully expected functionalities, so we still have several limitations. We will also introduce our roadmap and elemental technologies being necessary to overcome.KaiGai Kohei11:3001:00DMS 1120Writeable CTEsA Revolution in SQLlectureenIn SQL, Common Table Expressions are a way of creating and using views at run time, but only in read queries. Here we discover ways to use them in writes.
Find out about this exciting new feature! We can make Common Table Expressions even more powerful than the current SQL standard allows. We might even make it part of a future SQL standard.
Old syntax:
WITH [RECURSIVE] cte1 [(col[, col...])] AS (
READ_QUERY
[(UNION|INTERSECT) [ALL]
READ_QUERY
)[, cte2 ...]
SELECT ...
New syntax:
WITH [RECURSIVE] cte1 [(col[, col...])] AS (
ROW_RETURNING_QUERY /* for example, DELETE...RETURNING */
[(UNION|INTERSECT) [ALL]
ROW_RETURNING QUERY
)[, cte2 ...]
(SELECT|INSERT|UPDATE|DELETE) ...David Fetter13:3001:00DMS 1120Exploring Automated Join ConstructionAn discussion of HTSQL's join algebra and translation to SQLlectureenThis is advanced talk will be an in-depth discussion of HTSQL's
query language's design, focusing primarily on techniques used
for join specification and translation into SQL. It should be of
broad interest to core PostgreSQL core developers. It will likely
not be that useful for application developers or users of HTSQL
since the talk will be focusing on translation mechanics.
In the notes at the end of the SQL 2003 specification, there is a
suggested link feature (-> syntax) which could be used to relate
tables via declared foreign keys. It's been 8 years and this sort
of feature still doesn't exist in the core of modern relational
databases. It's not because the idea is bad, it's more likely that
it's because implementing it is non-trivial.
HTSQL has implemented this sort of mechanism in a manner that is
formally defined. This talk will provide an overview of what
we've done to get something like this working, the sort of internal
bookkeeping needed and lessons learned.
The talk will be given by Clark Evans & Kirill Simonov.
HTSQL is an high-level URI-based relational database query language
designed for accidental programmers. It is particularly good at
complex reporting and interactive dashboard construction. HTSQL
was continuously refined over the last 7 years with user feedback
and has become a very useful tool for not only casual database
users, but developers wishing to add ad-hoc query capability to
their application.
Clark C. Evans
HTSQL Website
Slides for 5/20
15:0001:00DMS 1120Using PostgreSQL for Flight PlanningHow PostgreSQL is being used at www.worldflightplanner.comlectureenPostgres is a powerful database, especially when coupling it with PostGIS to make it spatially aware. Using PostgreSQL and PostGIS allows World Flight Planner to provide advanced flight planning tools to both private and commercial pilots.
Using PostgreSQL and PostGIS allows you to access and modify your spatial data using SQL. This talk will provide an overview on the advantages to using PostGIS, how to properly store and query your geographic data, and offer some solutions to common problems.This talk will comprise:
- A primer on storing geographical data in postgres
- Everything you needed to know about projections, sphereoids.
- Cartesian Trig Vs. Spherical Trig
- Types of data we need to store (points, line strings, other geometries)
- Why we decided to use Postgres+PostGIS
- How we store global scale meteorological data (converting gridded data in binary format into rows in the postgres database for fast querying)
- Some of the challenges for storing aeronautical information in postgres.
- How the data was organized in the database (a huge challenge, as identifiers for airports and navigational aids are not globally unique)
- Odd airspace shapes (such as pie, or "pac man" shapes)
- Describe the overall relationships of aeronautical data (airports, navigational aids, fixes) with each other.
- Some custom functions we wrote to get the job done.
- Examples of how to use PostGIS functions to do fast computations, such as:
- Find me the closest airport to my current location.
- Does my flight path take me through restricted airspace?
- What heading do I need to take when I cross my next waypoint (bearings).
- Displaying geospatial data, How we did it (map server, mapnik, openlayers, etc..)
Blake Crosby16:3001:00DMS 1120Serializable Snapshot Isolation in PostgresMaking ISOLATION LEVEL SERIALIZABLE Provide Serializable IsolationlectureenSerializable isolation makes it easier to reason about database
behavior because concurrent transactions appear to have executed
sequentially. However, Postgres has historically only provided
snapshot isolation, even at the SERIALIZABLE isolation level, which
can expose anomalous states to queries.
This talk will describe a new SERIALIZABLE isolation level using
Serializable Snapshot Isolation (SSI). Unlike the typical two-phase
locking approach to serializability, SSI uses the existing snapshot
isolation mechanisms and retains most of their performance benefits.
It adds a new type of lock that's used to track read dependencies and
roll back transactions when dangerous structures appear in the
serialization graph, indicating possible anomalies.
We'll describe how SSI works and how we've implemented it for Postgres
9.1, including a new predicate lock manager. We'll explain what users
can expect when using the new SERIALIZABLE mode, and present some
performance results.Dan PortsKevin Grittner09:3004:00DMS 1110pldevsPL Developer SummitPL/ManiameetingenProcedural LanguageEevelopers will met and discuss current issues around PLsSelena Deckelmann10:0005:00Out and abouttouristTourist stuffSpend some time exploringotherenExplore OttawaOttawa has a large number of great attractions. Spend some time looking around and explore. Spend as much time as you want with us, or leave early. We will walk everywhere we go. Wear sensible shoes. Bring your camera. We'll probably have lunch somewhere along the way. Consider the weather (sun block, rain coat, umbrella, swim suit).
This is one option for Saturday. You are free to come with us or take part in all or some activities. Or make your own way around...
The agenda for the tourist day is completely wrong.
The forecast: http://www.theweathernetwork.com/weather/CAON0512
Cora's - 8 AM - Breakfast
- 179 Rideau Street - http://bit.ly/dngpnV
- if you arrive much past 8:30, you probably won't get
through in time to get to the next meeting point
- we will depart Cora's at 8:55
Rideau Center - 9:13
- south end of the Mall on the Mackenzie King bridge
- we are catching the #95 bus to direction Orleans to BLAIR 1B
- from there, at 9:27, take Bus route 129 (OC Transpo) direction
Aviation
Aviation Museum
- we'll do a guided tour first, then wander around
- departing here at 1:30
Earl of Sussex Pub
- arriving at about 2:15
- 431 Sussex Drive
Ottawa, ON K1N 9M6
(613) 562-5544
earlofsussex.ca
All times after we leave Rideau Center are very subject to change.
Dan Langille
National Memorial
Residence
Forum