From mid to late June credativ attended 3 consecutive conferences in Asia. The Open Source Summit Japan, the LC3 China and the PG Open China.
Open Source Summit Japan
As in the previous year, the Open Source Summit Japan was held again at the Ariake Conference Center on Odaiba, in the Bay of Tokyo. In order to cover the increasing number of participants, the event area was extended by several rooms. Accordingly, there was plenty of space for everyone.
In addition to a large number of technical presentations, mainly on containers, cloud and applications for automotive, there were also business and strategy orientated talks. All visitors could find topics of interest for themselves.
Specifically the presentation “Is There an Open Source Business Model: YES or NO?” by Jeffrey Borek (IBM) and Stephen Walli (Microsoft), deserves special mention, as it was the starting point for extensive discussions that day. Jeffrey and Stephen each gave their views and then gave the floor to the attendees to mix ask questions or give comments. The title being worded as it is doesn’t really explain the topic in detail. What the talk really was about is the question if a Open Source business model exists for large software companies.
Further to be emphasized is the presentation by our CEO Dr. Michael Meskes, whose question ”Is There a Future for Open SourceLC3 China LC3 China (LinuxCon + ContainerCon + CloudOpen) was held in Beijing at the China National Convention Center. In the immediate vicinity of the Olympic Green Olympia Park, which was built for the 2008 Olympic Games. Most of the presentations were held in English, although the vast majority of those attending were from China. A very welcoming gesture that definitely contributed to the international character. A large number of lectures with technical and business topics were held over the 4 days. Particularly interesting were the so called “office hours” as a separate position on the schedule. For this, 3 tables were provided at which discussions within the community could take place. On the whole, LC3 seemed to be a platform for China’s largest IT companies, but it also showed that hacker groups and student groups have their own place in the community. We were certainly very happy to be able to participate and are looking forward to the next year. Postgres’ growing success and the well-attended PostgreSQL® conferences, which have been held in China since 2015, led to the first PG Open China being held at the Hotel Nikko New Century Beijing this year. The aim of this conference was to provide a starting point for the Chinese PostgreSQL community and strenghthen its connection to the global PostgreSQL community . For or this purpose, an international committee was established consisting of Bruce Momjian, Joshua D. Drake, Oleg Bartunov, Ray Feng and Dr. Michael Meskes. After introducing this committee in a keynote, the event rooms were used for presentations and talks from the Chinese community. Joshua D. Drake and Oleg Bartunov also held a presentation. All lectures were translated with the help of an interpreter, so that all those present with less understanding of the English language nuances could comprehend everything. The number of visitors of the PG Open China fortunately exceeded expectations and as such that the growth potential can clearly be seen. Especially with the focus on connection to the international community, the conference will certainly grow significantly over the next few years. It is therefore highly probable that we will be back on site next year. We were very happy to be one of the fist to participate in the PG Open China. The second beta release of PostgreSQL 11 (which is now feature frozen) has been released recently. Time to look at some of the improvements that credativ has contributed in the area of checksums and backups. Since version 9.3 it is possible to enable checksums for the underlying storage of tables and indices during instance creation. Those checksums raise SQL errors if bit errors are encountered in their respective data pages, which allows for early discovery of storage issues. However those checksums are only verified if queries access the corrupted page. Running an explicit check is only possible with the forthcoming pg_verify_checksums application from version 11, however, it requires an offline instance in order to work. Our change allows for verification of checksums during base backups. This is a good opportunity to verify the checksum consistency as all data blocks needs to be read during a base backup anyway. Checksum failures are logged as warnings (rather than errors) in order not to abort the whole base backup when they occur. The commonly used pg_basebackup application was extended with the –no-verify-checksums option which disables the verification. The second change concerns the handling of replication slots by pg_basebackup during the setup of standby servers. Replication slots allow a primary to reserve the required transaction logs for the standby associated with the slot, even if it is temporarily down. Previous releases already allowed using a replication slot with pg_basebackup, however, this slot had to be created manually beforehand. Our change adds the new option -C or –create-slot and allows the on-shot creation of a standby clone including the usage of replication slots: Afterwards the standby just has to be started and will replicate automatically. In addition, several other small improvements to pg_basebackup and its testsuite were done by us. A patch that did not make it into the release is presented here nevertheless: parallel pg_dump to /dev/null in the directory format. The reason for it is the common usage of pg_dump to check for errors in a PostgreSQL instance where /dev/null/ is used a target in order not to use additional disk space. The problem is that /dev/null can only be used in the custom format which does not allow dumping in parallel. The directory format supports multiple concurrent processes but cannot use /dev/null as target as it is not a directory. Our patch adds support for /dev/null as a target when using the directory format. The reasons for the rejection were not technical issues with the patch but the fact that pg_dump is not a diagnostics tool and no special support for that should be included. Nevertheless, the submitted patch works and is being used by our clients. Versions of the patch for PostgreSQL 9.3, 9.4, 9.5, 9.6 and 10 are available. The credativ PostgreSQL® Competence Center has released a project roadmap for the PostgreSQL® Appliance Elephant Shed. Elephant Shed is a freely available PostgreSQL® solution developed by credativ, which combines management, monitoring and administration into one system. The project roadmap for 2018 includes following points: An additional planned feature is the implementation of REST API to control individual components. REST stands for REpresentational State Transfer and is an application programming interface based on the behavior of the World Wide Web. Specifically the PostgreSQL® database and the backup via pgBackRest should be addressed. Multi host support is also planned. A central control of several Elephant Shed instances is thus possible. In order to make Elephant Shed even more user friendly, various configuration parameters of the web interface are going to be adjusted. The project roadmap is of course also constantly being worked on. On GitHub you can leave us your feedback at any time. We would like to take this opportunity to thank all users and testers, and look forward to further development of the project! For further information please visit elephant-shed.io and GitHub. This article was originally written by Philip Haas Elephant Shed is now available as a Vagrant box. This makes it very easy to test and try out the PostgreSQL® appliance. Vagrant is an Open Source tool for creating portable virtual software environments. By being fully script controlled Vagrant makes it easy to generate virtual machines, in which a software component is installed for testing purposes. Vagrant itself is only the manager, whereas various backends such as Virtualbox or cloud providers can be used for the actual virtualization. For the development of Elephant Shed we have relied on Vagrant from the very beginning. This “box” is now also available in the Vagrant Cloud. To use this box Vagrant and VirtualBox must be installed. The host operating system hereby is irrelevant (Linux/MacOS/Windows), but inside the box runs Debian Stretch. The box is then automatically downloaded. This creates a virtual machine where Elephant Shed runs in a VirtualBox on your computer. We are looking forward to your feedback! Further information can be found on our Elephant Shed project page and on Github as well. The credativ PostgreSQL® Appliance is now available for download as a completely free Open Source solution. The new appliance called ‘Elephant Shed PostgreSQL® Appliance‘ offers a tremendous ease of use for enterprise PostgreSQL® operations. The long-term maintenance of the appliance will be handled by the PostgreSQL® experts of credativ. Prepared images for the major cloud and virtualization platforms are already in work and will be released soon for VMWare, Virtualbox, Vagrant and for the cloud platforms Microsoft Azure, Amazon Web Services and Google Cloud Platform. Elephant Shed builds on proven components, which are published exclusively under recognized Open Source licenses. These tools are an effective support for the management of a PostgreSQL® server. All components used are pre-installed and integrated into the integral automation system. The majority of these tools can be used via a comfortable web interface. For Elephant Shed, credativ offers comprehensive technical support with guaranteed service level agreements, which is also available 24 hours a day, 365 days a year. Support during installation and integration, as well as an introduction to Elephant Shed is of course also part of credativ’s services. Please do not hesitate to contact us if you have any questions about Elephant Shed and our service and support services. For more information, please visit our Elephant Shed- project page and Github. This article was originally written by Philip Haas.
PG Open China
Checksum verification during base backups
Replication slots during base backups
$ pg_basebackup -v -h primary.lan -D data --slot=standby1 --create-slot --write-recovery-conf
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 0/1D000028 on timeline 1
pg_basebackup: starting background WAL receiver
pg_basebackup: created replication slot "standby1"
pg_basebackup: write-ahead log end point: 0/1D0000F8
pg_basebackup: waiting for background process to finish streaming ...
pg_basebackup: base backup completed
$ cat data2/recovery.conf
standby_mode = 'on'
primary_conninfo = 'user=postgres passfile=''/var/lib/postgresql/.pgpass'' host=primary.lan
port=5432 sslmode=prefer sslcompression=1 krbsrvname=postgres target_session_attrs=any'
primary_slot_name = 'standby1'
Parallel dump to /dev/null
vagrant init credativ/elephant-shed
vagrant up
adminadminpsql -h localhost -p 55432 -U adminService and Support