The PostgreSQL Conference University of Ottawa Ottawa 2008-05-20 2008-05-24 5 Final - we hope 09:00 00:30 09:00 03:00 Fauteux 147B guc_tour Every option, explained Tutorial workshop en PostgreSQL now has 187 GUCs, most of which are server configuration settings, and begins to rival Oracle in complexity of administrator options. Take a high-speed tour of all of the options with core team member Josh Berkus. Just sit right back and you'll hear at tale, a tale of some GUCs ... PostgreSQL now has 187 GUCs, and begins to rival Oracle in complexity of administrator options. Most of these GUCs come from PostgreSQL.conf and session-level settings, and it's a rare PostgreSQL server where you want to accept all defaults. But with so many options, how do you decide what you want to change, and to what? Josh Berkus will go through every single option and explain it (at least breifly), giving advice and examples on many of the more important ones. At the end of the talk, we'll see how many options Josh can recite in one breath. Josh Berkus 13:00 03:00 Fauteux 147B oracle Tutorial workshop en Learn about the issues surrounding porting database applications to PostgreSQL. As many PostgreSQL applications begin their life by being ported from some other database management system, it is useful to look at what is involved in such an effort. This tutorial will teach what is involved in porting Oracle applications to PostgreSQL. Those who are interested in porting database applications to PostgreSQL but are not dealing with Oracle could still find most of this workshop useful, as the issues are quite similar in any case. Specifically, we will look at technical issues such as porting the schema, queries, functions and procedures, porting tools and modules, major philosophical differences between the products, as well as practical issues such as engineering and project management issues in database porting projects. This workshop is aimed at database developers and administrators, who want to learn about porting database applications to PostgreSQL or perhaps share their own experiences. Some experience with either Oracle or PostgreSQL is recommended. Peter Eisentraut 15:00 05:00 Royal Oak Pub registration The social way to register: at the pub Social other en Pick up your registration pack Stop by the Royal Oak Pub and get your registration pack. You'll help us avoid long line ups on Friday morning and you get to have a beer, and chat with your fellow attendees. We guarantee you'll spot someone famous. Dan Langille 18:30 05:00 Urban Well yahoo Join us for drinks, food, Yahoos, and Nintendo Wii. Let the games begin! Social other en Join us for drinks, food, Yahoos, and Nintendo Wii. Let the games begin! May 21, 2007 at 6:30 p.m. Urban Well, 244 Laurier Ave. East, Ottawa, Ontario Party on with Yahoo! and get your chance to win a FREE Wii! Come on down to the Urban Well for an evening of drinks, food and fun—and your chance to meet celebrated Yahoos like Donovan Schneider, Krishna Uppala. Find out what it’s really like to work on innovative products that impact 550 million people worldwide. Dan Langille 09:00 03:00 Fauteux 147B views How to simply leverage view materialization for even the most complex cases Tutorial lecture en Gain huge performance boosts for your applications with Materialized Views. Learn how to achieve 10-100x (or better!) performance gains from your bottleneck queries. Really. Complex queries repeated in application code often are recast as views. This makes programming easier, but it does nothing to improve the performance of the underlying query. When performance gains are needed, materializing the view is the next logical step. Views that are materialized are stored in tables rather than computed at request time, so joins, procedure evaluation, and conditions from the original query are pre-computed before queries are run against the materialized view. Because the data is in a table, performance can be further improved by adding indexes. The difference between selecting a single row out of an indexed table or computing it in real-time from a dozen joined or aggregated tables can be staggering. Many developers shy away from materializing views in Postgres because the functionality is not built in, as it is in Oracle. It can take a lot of meticulous work to build by hand the stored procedures and triggers necessary to keep the materialized view up to date. It turns out there are a variety of materialization techniques you can apply in Postgres that aren't available in Oracle's materialized views, so you're better off building them yourself after all. The time you put in enhancing your database's performance is well worth the orders of magnitude of speed enhancements you will enjoy as a result of your labor. In this tutorial, I will present a straightforward approach that can be applied to materialize any view. I'll also present two new techniques that have not yet been described for materialization in Postgres. The first shows how to implement eager materialization of a view with a time dependency (e.g., a view based on 'now()'). The second technique shows how to implement lazy materialized views in Postgres. Lazy materialized views are like their eager counterpart, but only one refresh per transaction is guaranteed, making materialization viable for aggregate and other one-to-many views. Both of these problems are currently unsolved in Postgres, as well as in Oracle's built-in view materialization. After this tutorial, you will be able to materialize any view and see HUGE performance gains. Dan Chak 13:00 03:00 Fauteux 147B patterns Everyone else is doing it wrong, why can't we? Tutorial workshop en Many common solutions used in the world of database design go beyond sub-optimal to downright awful. Many times these pattern are carried over from early years of database design work, or learned from working on poorly implemented solutions. This talk we discuss many of the common faux pas, focusing not only on the bad, but hopefully providing better solutions to use in their place. We have all seen certain patterns used over and over again in database design, but sadly many of the common ones tend not to be best practice. In this talk we will cover many of the most common database "anti-patterns", highlighting why they are bad, talk about ways to overcome them, and maybe even note a few cases where (gasp) you might have to use them. Robert Treat 10:00 01:00 A projectman Why there is no traffic on replica-hooks-discuss Community lecture en This is a talk about PostgreSQL feature development as observed over the past 7 years. Some features have been developed carefully, others have grown somewhat organically, and in some areas features continue to be an area of controversy. I explore the different strategies people take to developing these features, and make some suggestions on how the community might organise some future large feature development. Replication is a topic of perpetual discussion in the community: how it is designed, how it is integrated into the back end, and whether a given approach is the right one for any given job. Every three or four months, the topic erupts on some list or other, invariably with a suggestion by someone that some approach is "obviously" the right way to do it. These discussions always include the observation that replication is a big topic that has many applications, and the idea has always been to provide more than one facility, to satisfy the various kinds of applications. This is consistent with what several other DBMS do; and there is no evidence that a single system is on the horizon, which can do everything for everyone. In 2006, frustrated by the poor results from these occasional discussions, I tried to set up a PG foundry project that would come to some agreement on the minimum changes needed to the main tree for various replication systems. I reasoned that, since there are several different projects, each attempting to deliver replication, we had enough experience to be able to draw a first pass at an API. Then we'd be able to come back to the community and see whether there was real interest in that API. This approach has more or less completely failed: there were some early postings by a couple different projects, but most replication system designers rejected the list as a complete waste of time, and the idea of designing an API was dismissed as burdensome and complicated for the replication system designers. I have concluded that this result was predictable, because of the ways features are developed in the community. This talk outlines the ways I think feature development actually happens, explores some of the costs and benefits of that development model, and tries to suggest some ways that larger, more complicated systems will be possibly developed in the face of the actual development strategies in use. I first outline what the official approach is: come up with an idea, discuss in the community, find consensus, then implement. I then discuss some kinds of strategies more often really seen. The first is the single-source proposal that emulates the official approach. These are very often improvement proposals: "Here is a chunk of behaviour that is missing or wrong." This conforms nicely with the TODO-list system we have. The second is a more complicated proposal: the code is already written, or nearly done. This approach is far more common that is usually acknowledged, and it sometimes leads to fractious discussion. The third is a degenerate version of the second: some filthy hacks have been applied by someone, and they come with a proposal to fix up the hacks and get them integrated in the code. The fourth is the frustrated patch: when consensus can't be achieved under the first approach, some developers go away and develop a patch that implements the proposal anyway, in the hopes that running code will be convincing. Each of these has a number of costs to the community. None of them really allows for large community planning of multi-release feature development, which is why bigger features are sometimes hard to discuss. Andrew Sullivan 11:30 01:00 A logic Answering complex questions & migrating data Applications lecture en This talk focuses on two common data management tasks, and how the relational model can help. First, how to answer complex questions about your data, and provide an answer that can be understood by non-IT people unambiguously. Second, how to effectively extract meaning from disorganized (or under-documented) data sets for effective data migration or reporting. The relational model, on which all SQL DBMSs are based, is founded on both logic and sets. Relational operators like JOIN and UNION have a direct logical meaning, and you can use those operators to answer complex questions from your data unambiguously. Not only can you provide unambiguous answers, but you can translate SQL queries into the language of logic, which can be understood by people outside of IT. Equally important is the utility of logic working backwards to help find meaning in disorganized (or perhaps undocumented) data sets. Nearly every inherited data set is less than perfectly documented, and often inconsistent. Changes to (or replacement of) the application require this data to be migrated. By iteratively making assertions and then testing them using the rules of logic and the convenience of sets, you can find exceptions and contradictions that help you refine the meaning of data, and possibly correct inconsistencies. Jeff Davis 13:30 01:00 A drugs Secure enough for law enforcement DBA lecture en Do you have a large amount of data that needs to be searchable, aggregated and extremely secure at the same time? See many of the creative solutions that have been deployed to help facilitate how we put PostgreSQL to the task of drugs. PostgreSQL is known to be a powerful open source relational database with many uses. One such use is warehousing EMRs (Electronic Medical Records) from oncology practices across the country. PostgreSQL, Perl, Apache, Ubuntu Linux and OpenBSD are all used for their strengths to deliver information to pharmaceutical companies to see what their drugs are doing for individuals in real world scenarios. Do you have a large amount of data that needs to be searchable, aggregated and extremely secure at the same time? See many of the creative solutions that have been deployed to help facilitate how we put PostgreSQL to the task of drugs. Aaron Thul 15:00 01:00 A nasa Satellite Science Data Processing with PostgreSQL Applications lecture en Over the last several decades, NASA has been one of the major collectors of Earth climate observations from space. The quantity of data from satellites has been exploding with the current generation producing hundreds of terrabytes of science data products each year. The tools used to process this vast quantity of data have evolved over the years as well. This talk will include an overview of science data processing and discuss the evolution of the ground system processing from large scale proprietary Unix platforms and traditional commercial databases to Open Source solutions including commodity clusters of Linux machines, Perl based software frameworks, and PostgreSQL. It will discuss how we use PostgreSQL to track metadata and provenance throughout the data processing. Curt Tilmes MODIS Web Page MACUV Web Page (OMI data) 16:30 01:00 A search Using full text search for websites and mailing lists Advanced Features lecture en search.postgresql.org uses PostgreSQL Full Text Indexing to search mailinglists and websites. This talk will outline how it was built and deployed. This talk will discuss how search.postgresql.org uses the PostgreSQL Full Text Indexing, first tsearch2 in 8.2 and now the native Full Text Indexing in 8.3, to deliver searches of web and list archives with much better performance and capability than the previous generic web indexing tools used. It will discuss how the system is built, how context sensitive indexing helps a lot in keeping performance and results good, as well as how the full text searching feature in PostgreSQL was set up. Magnus Hagander 17:30 01:00 A accel A case study Horizontal Scaling lecture en Abstract: As the amount of data in the database grows more and more it becomes necessary in most of the applications using database to organize it in a way which is optimized for both high data insertion rate and faster query execution. This paper presents a case study which describes a database design which uses data partitioning to store huge data and a lightweight multi-threaded application which works as a middleware for faster query execution (up to 13 times). More emphasis is given on the multi-threaded application which executes queries spanning across multiple tables faster than only one single-threaded Postgres process. Approximately 90GB of data is going to be supported by this implementation with queries spanning across entire data. I will speak about the design of the multi threaded application which is developed and tuned as per our project needs to get the results from the database faster than it would actually return when asked to do it by itself. Initial tests show the performance (in terms of time taken to get the query result set) gain obtained is ~13 times more. Data size and H/W specification: ~90 GB of data spread across more than 20 tables is stored in the database. The tables are partitioned on day basis and inheritance is used to enable easy maintenance of data. Each tables contains anywhere between 20,000,000 to 40,000,000 records. The H/W used to run the application is 2 processor machine with 2 hard disks, 2GB of RAM 4GB of swap space (Sun-Fire-V245). Need for speed: The data is continuously flowing into the machine at a very high rate and is available on the machine's local hard disk for processing. This data is then processed and entered into the DB in appropriate tables. The database is expected to be up 24x7 because the data coming from the network needs to be stored continuously without loss. Database can do it all? No: High data insertion rates are managed by Postgres with its default configurations and some minimum tuning of configuration parameters. However, we face following problems/limitations with using Postgres for querying: 1. Time taken to execute the application specific queries is very high since queries span across multiple tables with huge size and involve ‘GROUP BY’ and ‘aggregation’ operations. 2. For data spanning across more than 3 tables, Postgres throws ‘out of memory’ error when the Postgres process size handling the query in memory goes beyond 4GB. 3. For querying on large data sets significantly more ‘shared_memory’ and ‘work_memory’ is required. I will explain how this application tries to overcome all the above mentioned problems. Durgaprasad S. Pawar My brief profile 09:00 01:00 B opening Welcome back to PGCon and the Path to the Future Plenary lecture en After a greeting by Dan, Bruce will consider possible future directions for the PostgreSQL software and the project. Bruce will highlight ways in which the future direction of PostgreSQL will be different from the challenges of the past. Future changes include moving beyond feature parity with other databases, and focusing on performance improvements for single-session workloads. Finally, we can now see a day when PostgreSQL will be a database leader, which will bring new challenges. Dan Langille Bruce Momjian 10:00 01:00 B dbilink Federating Heterogeneous Data Stores - Using speadsheets and cvs files with databases Advanced Features lecture en Have you ever needed to get data from a spreadsheet and put it in a database? Have you ever wanted to do real queries on a CSV file? DBI-Link 3.0 makes these operations easy. For years, DBI-Link has been used in large production systems for communicating between Postgres and other data sources, not just other RDBMSs. Find out what's new in the latest version, and make your migration easier. David Fetter DBI-Link Software 11:30 01:00 B archives Archival of digital data Horizontal Scaling lecture en The Hitachi Content Archive Platform (HCAP) is a storage system designed for the archival of digital data. HCAP software runs on a cluster of Linux nodes, implementing a shared nothing architecture. File metadata is kept in a set of Postgres databases, one running on each node of the cluster. The data managed by Postgres is partitioned into "regions" and copies of each region are kept on multiple nodes: one master, and zero or more slaves. Typically there are 32-256 regions, depending on the cluster size, and two copies of each region. Region copies are kept synchronized using a homegrown replication scheme. Before an update is reported as successful, the master copy update is committed, and updates to slaves are acknowledged as received (but not necessarily committed). The paper and talk will focus on how data and access are partitioned, how replication works, how data integrity is maintained across failures, and how the temporary and permanent losses of region copies are dealt with. Jack Orenstein 13:30 01:00 B xbase Migration of existing Xbase++, Clipper, FoxPro, Visual FoxPro or dBase applications to PostgreSQL Applications lecture en An introduction into the Xbase++ language and how it supports the PostgreSQL server. This lecture is intended for programmers which have Xbase++, Clipper, FoxPro Visual FoxPro or dbase applications in the field. An introduction into the Xbase++ language and how it supports the PostgreSQL server on the Client and Server side. The first part of this lecture is intended for programmers which have Xbase++, Clipper, FoxPro, Visual FoxPro or dbase applications in the field and want to migrate them to a Client/Server technology. In the second part the integration of Xbase++ as a new language for stored procedures of the PostgreSQL server is outlined. Attendees will get a basic understanding how easy the migration of existing even large scale solutions to the PostgreSQL server can be. Finally user of other development tools and languages see how the Xbase++ language for stored procedures can increase productivity in various ways. This includes also the ability to extend the stored procedure language with your own domain specific syntax. The presentation slide can be downloaded from here http://www.alaska-software.com/products/arctica/xbase++-meets-postgresql.pdf Steffen F. Pirsig 15:00 01:00 B sync High availability solution to minimize downtime Horizontal Scaling lecture en NTT has developed a shared-nothing replication system for PostgreSQL implemented with transaction log shipping and Heartbeat. The goal is minimizing the system downtime and the impact for update performance. Failover can be done within 15 seconds and the overhead is at worst 7% on heavily-updated workloads in the current implementation. The replication solution realizes 99.999% availability so that it is applied to production systems. We will explain advantages of the solution and future direction of the development. The goal of the synchronous log shipping replication is minimizing the system downtime and ensuring 99.999% availability. At the same time, we need to avoid performance degradation on heavily-updated workloads and functional restrictions. We chose synchronous log shipping approach to accomplish the purposes. There are many replicators already, but almost of them are asynchronous or have some restrictions derived from statement-based approach. We think log shipping is the best solution for high availability and extensibility in the future. We implemented basic functionalities of the replicator on PostgreSQL 8.2 with a WAL sender through network. It works like the standard warm standby configuration for now, but can minimize the delay of transfering logs. The primary server can commit only after the commit log is transferred to a standby server. After a failure of the primary server and switching to standby server, the standby server has already received all of the transaction logs and doesn't need to read the last WAL records from the primary's disks. We can use shared-nothing disks and create a high availability cluster with inexpensive storages. Even if shared disks are available, it makes possible not to have to wait for ummounting the disks from the primary server. There are two remaining works; more speedy failover and load-balancing. The delay on failover is restricted by recovery speed in the standby server. Recovery performance is improved in 8.3, so we have a plan to port our replicator to 8.3 and to measure the performances. The next step is load balancing. If read-only queries are allowed on standby servers during a PITR, the replicator will have load balancing capabilities with little or no delay. ITAGAKI Takahiro 16:30 01:00 B hot The Technical Architecture Hacker lecture en This session will cover the technical architecture of HOT (Heap-Only-Tuples), a key feature of PostgreSQL 8.3. Prior to HOT, frequent updates on a table would bloat the table and the indexes on it, negatively impacting long-term database performance. While frequent manual or automatic vacuuming could repair the table/index bloat, the same vacuuming caused performance problems by putting additional load on the database. HOT now attempts to address these problems by reclaiming the space used by the old, dead tuples without requiring vacuuming of the table. Pavan will discuss the internals of HOT and the interesting technical challenges faced during HOT's development. He will also share HOT’s performance statistics and its limitations. Pavan Deolasee 17:30 01:00 B lightning Everyone gets 5 minutes of Fame! Community lecture en The always enlightening, sometimes frightening, collection of talks from around the PostgreSQL community. You get 5 minutes to say your spiel, dazzle the crowd, and reach for glory. Robert Treat 10:00 01:00 G java Manipulating complex data models using component or object oriented methods Applications lecture en The Java Enterprise Edition specification defines APIs that allow developers to manipulate complex data models using component or object oriented methods. This session will take a look at how Enterprise Java Beans (EJB) interact with PostgreSQL through the Java Database Connectivity (JDBC) driver and what types of queries that are generated using the Enterprise Java Beans Query Language (EJB/QL) or the Java Persistence API (JPA). Last the session will explore how to optimize the throughput for the application by optimizing the load strategies using the latest versions of PostgreSQL and the JBoss application server. Technology: ----------- PostgreSQL 8.3 and JBoss Application Server 5.0 Jesper Pedersen 11:30 01:00 G inverted Multicolumn index support and speedup of index update Hacker lecture en We present new multicolumn feature support of the Generalized Inverted Index (GIN) and intermediate results of our research on fast online GIN update. Generalized Inverted Index (GIN) is a popular index, which used to speedup text search and set operations. However, it currently lacks support of multicolumn index support, which limits its usefulness. We plan to add multicolumn support to the GIN for the next release of PostgreSQL. It's known that inverted index is fast and well scaled but has slow index update compare with Btree and GiST indexes. This is a well known problem of inverted index not specific to the PostgreSQL and is a great challenge we want to attack. Current GIN implementation supports fast bulk create index and we hope to reach comparable speed for online index update. Teodor Sigaev Oleg Bartunov 13:30 01:00 G mysql The learning continues Community lecture en While PostgreSQL was busy becoming the most advanced open source database, MySQL became the most popular. Join former MySQL community manager Zak Greant for a discussion of what has helped MySQL succeed in developer communities and what the PostgreSQL community could learn from it. Susanne Ebrecht 15:00 01:00 G feedback Using customer feedback and benchmarks to reduce performance and productivity loss Hacker lecture en As multi-core systems and storage space become cheaper, PostgreSQL is increasingly hard pressed to scale to the size of servers available to most users. Sun's performance engineers use customer feedback and industry standard benchmarks like TPC-E and SpecJAppserver to examine what causes these issues and what to do about them. Sun has a small performance engineering team devoted to PostgreSQL scalability. For the past year, this team has been working on making PostgreSQL scale to more cores and larger databases, both to satisfy customer demand, and to improve benchmark scores. This session will cover their findings on PostgreSQL scalability and suggest ways to improve PostgreSQL 8.4 in order to make it scale to twice as many cores and terabytes as 8.3 did -- as well as beat the scalability-destroying TPC-E benchmark. Areas covered: * Setup,loading, utilities * ProcArrayLock problem for OLTP workload * Load, Index Build, Backup Restore for DSS Workloads * Discuss ideas on what can be done Jignesh K. Shah 16:30 01:00 G pgpool2 Horizontal Scaling lecture en Tatsuo Ishii will show and explain how to use pgPool2, current project status, and planned features. Tatsuo Ishii 19:00 05:00 Velvet Glove dinner PGCon Dinner Social other en EnterpriseDB would like to invite the pgCon 2008 attendees to join them for an evening of food, drink and entertainment at the Velvet Room on the evening of the 22nd May. Location: Velvet Room Address: 62 York Street, Ottawa Start time: 7PM Dan Langille 10:00 01:00 A postgis A Standards Based Geographic Extension for PostgreSQL Advanced Features lecture en A introduction is provided to PostGIS, a PostgreSQL extension for managing geographic feature (vector) data. A particular focus is provided on the OGC Simple Features for SQL standard from the Open Geospatial Consortium, which is implemented by PostGIS, and how PostGIS fits into the standards based approach to managing geospatial data. A brief review of applications supporting PostGIS will also be provided. Frank Warmerdam 11:30 01:00 A listen How they work, and how they don't Applications lecture en A description of how notifications currently work. LISTEN and NOTIFY commands. New features desired: guaranteed multiple delivery, and event payloads (associated messages). Need to use shared memory as opposed to using a table. Description of plan for new implementation. Andrew Dunstan 13:30 01:00 A dracones A PostgreSQL/MapServer WebGIS application for Public Health Surveillance Advanced Features lecture en We make extensive use of PostgreSQL and its spatial extension PostGIS. We use this for the development of a Web application/framework for Public Health Surveillance, with two goals: (1) to provide a way to manage and visualize spatial information about cases of disease in a flexible and portable manner, and (2) to provide a way to easily connect to external statistical and analytical software (in particular we make use of the SaTScan software). Christian Jauvin 15:00 01:00 A htsql An innovative URL-2-SQL translator for Web 2.0 applications. Applications lecture en With the proliferation of web-based applications and increased user technical ability, distributed applications with multiple front-ends becomes a viable, if not essential approach to software construction and deployment. This talk will describe an approach to web-based databases, specifically a URL-2-SQL translator which converts common assumptions about how web resources should work into SQL queries for a PostgreSQL backend. Interestingly enough, with a tool such as HTSQL together with the maturity of database-level constraints and stored procedures, the pendulum favoring 3-tier solutions swings back to a 2-tier approach. HTSQL is a mature, and competently developed URL-2-SQL translator; it's installation essentially publishes an entire PostgreSQL database "on the web" with little effort. HTSQL intelligently maps URLs onto SQL queries, using information_schema for configuration (with manual overrides if needed), putting most databases on-the-web in less than a few hours. A bulk of SQL constructs are covered, permitting standard DML (and via work-in-progress DDL), and enabling a vast majority of complicated queries. HTSQL has been field-tested with over 5 years of refinement, through 3 complete rewrites; and is deployed at medical research laboratories at UofM, UIC, Harvard, Yale, UCLA, Emory, U Washington, and many others. This talk will cover the following items: * A general discussion of web querying (by Clark Evans) * The vision for HTSQL, a preview of HTSQL and its regression test schema (5 min) * Key Concepts for HTSQL (10 min) - Path segments, commands, REST - Locators, aka Primary Key lookup-expressions - Selectors, columns, join specifiers, functions and the like - Projections and Aggregate Functions - Filters and Formatters * Configuration and End-User Details (5 min) - Perspectives (aka Roles) and Aspects (aka Tables) - Use of information_schema to infer relationships and details - Example configuration to "htsql-enable" Trac - Example queries on Trac - Making a plugin via Python * Implementation Goodness (by Kirill Simonov, 25 min) - Syntax Definition and Parser - Binding of query to meta-data - Internal query representation and macros - Translation to SQL - Handling aggregation and projections - Details on plugin-handling - Security concerns * A generic Web 2.0 GUI, "DBGUI" using QooxDoo+HTSQL (by Alexei Golovko, 5 min) * Summary, Q&A, and Conclusion (by Clark Evans, 10 min) This is a very technical discussion. Moderately complicated SQL will be used, regularly and without apology, in order to illustrate key concepts. Clark Evans Main Website, Specification and Code 16:00 01:00 A npgsql A .Net Data Provider for Postgresql Applications lecture en This presentation will introduce Npgsql, a .Net data provider for Postgresql written in 100% C# code. Npgsql allows programs written for .Net Runtime to access postgresql database servers. It can be run on GNU/Linux, Windows, MacOS and any other platform which has a .Net runtime available. This presentation will show what is Npgsql, history, features and code samples which demonstrate what can be done with it. Also, will be presented what is expected to be seen in future releases. Josh Cooley Hiroshi Saito Francisco Figueiredo Jr. 10:00 01:00 B windows Integrating with Windows and Active Directory DBA lecture en Many larger networks are based around Windows and Active Directory. This talk will show how PostgreSQL can integrate and work with such existing environments. This talk will outline some of the challenges to overcome when installing PostgreSQL in a Windows enterprise environment. Examples include mass-deployment, Active Directory integration for authentication and authorization and data exchange with existing systems. Both deploying the Windows version of the PostgreSQL server and integrating a Linux/Unix PostgreSQL in existing Windows-based infrastructures will be covered. Magnus Hagander 11:30 01:00 B irods RDBMS-based Data Grid, Persistent Archive, Digital Library Horizontal Scaling lecture en The integrated Rule Oriented Data management System (iRODS), is an open source software system that implements data grids, persistent archives, and digital libraries. iRODS incorporates an RDBMS for storing and querying persistent information and a distributed Rule Engine to invoke micro-service workflows. This session will present an overview of iRODS, how it makes use of PostgreSQL, and a comparison of PostgreSQL and Oracle iRODS instances. The integrated Rule Oriented Data management System (iRODS), is an open source software data management system being developed by the Data Intensive Computing Environments (DICE) group at the University of California San Diego with funding from the National Archives and Records Administration and the National Science Foundation. iRODS is the "Next Generation" product from DICE, building on the expertise developed through production use of the Storage Resource Broker (SRB). The SRB organizes more than 2 petabytes of distributed data into shared collections on a world-wide basis. iRODS 1.0 was released in January, 2008. iRODS functionality includes: maintaining global name spaces for identifying files, users, and storage resources, authentication and authorization controls, high performance WAN data transport, system and user-defined metadata, query-based data discovery and browsing, management of data distribution and replication, and checksum and synchronization mechanisms. iRODS provides a flexible, adaptive, and customizable data management architecture through the use of a custom-designed Rule Engine at its core which invokes 'micro-services' that are integrated into workflows to process requests and handle information. iRODS automates the execution of management policies by enforcing rules directly at each storage resource. Metadata, both system-maintained and optional user-defined, are ingested and accessed in an RDBMS system, either PostgreSQL or Oracle. Accesses go through the iRODS clients to iRODS servers via the iRODS protocol, and then to the catalog library which interfaces to the RDBMS via either ODBC or OCI. Performance and scalability are critical, as the digital holdings can be massive in size, measured in hundreds of millions of files and petabytes of storage, and may be maintained for decades. Wayne Schroeder 13:30 01:00 B plproxy Paritioning databases and using remote calls Horizontal Scaling lecture en There are several ways how plProxy can be useful in case of large sets of online data. It was designed for partitioning databases but now we have taken it into use also for remote calls. pgBouncer is natural part of plProxy installations. I would like to describe some growth paths for a company and its databases. In the beginning it's usual to have one server and one database. Depending on success you may end up with any number of database servers. Load balancing with pgBalancer, pgBouncer and plProxy Using plProxy for remote calls Using plProxy for Partitoning Asko Oja 15:00 01:00 B pqg how it is implemented, various uses Advanced Features lecture en Queues in database are rarely used, because thus far is has been impossible to implement both robust and high-performance queue in general-purpose RDBMS. In PgQ we managed to solve this by generalizing snapshot-based event tracking first introduced with rserv, now used in Slony-I. This presentation will describe the implementation and also real-life use cases. - how to implement queue with generic SQL, problems that appear - txid/snapshot based approach - producing events (various helper functions) - consuming events (core API) - tracking events (general, by-event, by-batch, pgq_ext helpers) - use-case: email sender - use-base: moving data - use-case: replication - future: sets, distributing queue over several machines Marko Kreen 16:00 01:00 B bucardo The open source multi-master replication system Horizontal Scaling lecture en Learn the strengths and weaknesses of Bucardo, an asynchronous multi-master replication system, and see a demo of it in action. Bucardo is a new multi-master and master-slave asynchronous replication system for Postgres. It uses Perl, PlPgsql, and PlPerlu to get the job done, and has many other features that can make moving your data around from one server to another a lot easier. We'll look at quick look at the problems involved with replication in general, and then look at how Bucardo approaches them, both in normal master to multiple slaves configuration, and in the master to master configuration. We'll examine the many custom hooks that Bucardo allows you to write an insert into any point in the configuration. Bucardo spurred the development of a helper module, DBIx::Safe, which prevents said hooks from doing things they should not, while still having deep access to the databases. Greg Sabino Mullane 17:00 01:00 B closing Good bye and thanks for all the fish Plenary lecture en Closing Session Good bye, and thanks for all the fish. Charity auction of select swag. Dan Langille 10:00 01:00 G searches Similarity searches in Postgresql using metric spaces Hacker lecture en A way to realize similarity searches between objects as words, sounds, images, etc.. using metric spaces. It is a way to do range queries, nearest neighboor queries and K-nearest neighboor queries. Includes coverage of the "Edit Distance" contrib module for PostgreSQL, as well as external indexing in C, and ongoing work to construct a special index type for similarity search. Enrico Pirozzi Pg Foundry project page Last Italian Presentation 11:30 01:00 G sepostgresql "System-wide consistency" in access control DBA lecture en SE-PostgreSQL is a security extension built-in PostgreSQL, collaborating with SELinux. The purpose is to achieve "system-wide consistency" in access control, with applying fine-grained mandatory access control to various database objects such as tables, columns and tuples. This feature enables to deploy a database management system within information flow control scheme integrated with operating system. It helps protect our information asset from some of threats like leaking, defacting and destruction. This talk introduces the background, features and implementations of Security-Enhanced PostgreSQL (SE-PostgreSQL). PostgreSQL's GRANT system offers extensive flexible access control. However, it can be difficult to integrate this with a uniform, centralized security policy for a secure operating environment. Even if initial permissions on database objects are made consistent with the host system's security policy, maintaining consistency despite policy changes through manual updates by the DBA is prone to numerous failures. The situation becomes even more untenable if the DBA has restricted privileges. Government security agencies are demanding a way to enforce uniform, top-down security policies across all of their information systems, including (and especially) databases. SE-PostgreSQL does this for PostgreSQL, allowing our database to be used for their purpose. SE-PostgreSQL is a security extension built-in PostgreSQL. It works collaboraing with operating system (SELinux), and enables to provide fine-grained mandatory access control. When a database client send SQL queries, SE-PostgreSQL checks them and makes its decision refering the SELinux's security policy about whether these queries should be executed, or not. This check is never avoidable for any client including privileged one, and is also applied at column/tuple level. SELinux applies its security policy to check any system call invocation. The key concept of SE-PostgreSQL is applying a single unified security policy for both OS and RDBMS. It will make same and consistent results, even if our information assets stored in different medias. We call the feature as "system-wide consistency" in access control. This feature is a piece of information flow control scheme, typically required by extreme robustness environment. KaiGai Kohei SE-PostgreSQL Project Home http:// 13:30 01:00 G pgrelease What Makes the Project Work Community lecture en Core team member Peter Eisentraut gives a partially behind-the-scenes look at the processes that make the PostgreSQL project work. The PostgreSQL project is founded on the principles of open source and an open community. Nevertheless, there are many processes in the routine of the project that are not obvious to all of the public: How are releases managed? How are release notes and press releases prepared? How are downstream packagers involved? How are bugs tracked? Who has commit access? Who manages the project's servers? How are security issues handled? What does the core team do? What is SPI and how are PostgreSQL's funds managed? I have worked with, tracked, and managed many of these issues over the years and would like to give the wider community an overview of what is happening in the project and perhaps point out new ways in which people can get involved. Peter Eisentraut 15:00 01:00 G performance Results from different OS and FS DBA lecture en In this talk, we will present a detailed analysis of the scenarios tested (different loads and number of users). Furthermore, measurements reflecting improvements in the newer PostgreSQL version will be summarized. How do we tune PostgreSQL? There is no cake recipe. The default configuration is far from being good to a high-end hardware. The problem is that it's hard to find a good configuration for different database designs. Thinking about it, we did a research aimed to test dozens of available operating system and DBMS parameters, and different filesystems. It was used 64-bit multiprocessor Linux plataforms and a fiber channel SAN. In this talk, we will present a detailed analysis of the scenarios tested (different loads and number of users). Furthermore, measurements reflecting improvements in the newer PostgreSQL version will be summarized. Fernando Ike some pgbench test results (it's not all of them) 16:00 01:00 G upgrade Upgrade on-the-fly or with minimal down time. Hacker lecture en Upgrade from old PostgreSQL version to the new is now possible only through data export and import. PG upgrade is a project which should allow to upgrade database on-fly or with minimal down time. Database upgrade is not easy task. Session will show current situation and it will focus on each affected parts of PostgreSQL and its conversion process. Presentation will start with storage module. Difference between page versions will be described include a way how to convert page structures and tuple headers. Second very important part is system catalog. Each new version of postgreSQL usually has modified system catalog structure. Catalog contains very important data for postgreSQL run and its conversion is very complicated. Talk will show some possible ways. Data types on disk structure conversion is last big part of upgrade process which will be presented. It also include varlena encoding change. Zdeněk Kotala Project Wiki page 18:00 05:00 Royal Oak Pub fripub Last chance for social intercourse before the Touristy stuff tomorrow Social other en The last big social event... Be there or miss out. :) Dan Langille 09:30 05:00 Out and about tourist See the sights in and/or around Ottawa Social other en Play Tourist We will organize a visit to one of the many local tourist attractions. Dan Langille