The core team of QGIS strives hard to provide the most advanced and user friendly GIS for free use by everyone. In the core QGIS project, every line of code that gets committed is subject to peer review when contributed by a non core developer. This gives us an opportunity to identify and correct inadvertent (or intentional) security issues that a developer may introduce into the code base. By contrast, all of the plugins that are published via the QGIS plugin repository are reviewed by the plugin developers themselves and we don’t have good insight into how much due diligence is applied to plugin code management.
The vast majority of our plugins (listed in http://plugins.qgis.org/ and inside your copy of QGIS) are developed by third parties, either individuals, companies, and institutions. As such, they are outside our direct control and the developers often relatively unknown to the QGIS community. We view this as a potential security risk. We are convinced the risk is small, because of many factors including the “many eyes” principle (the code is visible to everybody, and in use by thousands of people), but cannot exclude the possibility that someone tries to inject malicious code into a plugin.
In order to address this situation, we looked into the opportunity of implementing automatic tools to scan plugins, before their publication, and spot potential problems. Our research indicated that this approach would be difficult and costly, and easy to circumvent.
We (the PSC) therefore decided to implement a simple yet robust approach to security, based on the ‘web of trust’ principle: we trust people we know well in the community. You will see on the http://plugins.qgis.org web site that there is a ‘Trusted Author’ tag has been applied to plugins created by those members of the community that we know and trust.
The criteria for ‘Trusted Authors’ includes those community members that regularly meet at our QGIS developer meetings, and and those that are in almost daily contact with the core team via our developer mailing lists or background project discussions. The remaining plugins (and there are wonderful, reliable, robust, and useful plugins in the list) have not been given the ‘trusted’ label.
We would be delighted if a side effect of this choice would be to stimulate more active and direct involvement of plugin developers in the QGIS community. All plugin developers are therefore invited to join us at one of the next developer meetings (AKA HackFest), or otherwise become a recognized, active member of the community, so they can be integrated as ‘trusted’ plugin developers.
(Let me know if you have links to other language versions which were not sent to the mailing list.)
Looking at the English report, most responses were filed by regular (49.7%) and advanced users (35.9%) who use QGIS at least several times per week. One interesting result is that responders feel that the project should still prioritise new features when spending funds:
Top 3 “highest priority for spending QGIS funds”
Important features that are missing (50%)
More bugfixing (24.1%)
Improved user documentation (12.4%)
This is also confirmed by the free comments section were roughly 23% of responders were asking for new features, 19% called for more stability (fewer releases and new features), and 9% for better documentation.
Documentation improvements were followed closely by calls for a more structured approach to plugins (making it easier to find the right tool for the job), stricter plugin documentation requirements, consolidation of plugins with similar functionality, and integration of key plugins into core.
When interpreting these results, it’s important to keep in mind that responses are skewed towards experienced users, who are more likely to require specialist functionality. Beginners on the other hand might rank stability, ease of use of core functionality, and good documentation higher.
We are pleased to announce the first round of funding for the QGIS grant programme.
What is the grant programme?
The QGIS.ORG grant programme is our way to accelerate and streamline development of the QGIS.ORG project by rewarding committed developers and contributors for their work through a grant system. It is a way to distribute our funds amongst our team members in a fair and transparent way.
Why have a grant programme?
There are four main reasons for embarking on a grant programme.
The first intent of the grant programme is to amplify the contributions of grantees by allowing them to spend more time on QGIS over and above what they would be able to do on a purely volunteer basis. At the broader level we would also like to avert the potentially negative reaction to funded development work in QGIS: “Why should I donate my time to work on QGIS when others are paid to do it?” And rather create an aspirational environment: “If I make a large contribution to QGIS I could also be eligible for a grant like other dedicated contributors have received.”
To simplify the decision making process for how to spend the funds received in the QGIS project via our Sponsorship and Donations programmes. The grant programme would allow us to streamline our decision making when it comes to funding developers. We receive many proposals for funding various activities in QGIS which invariably lead to protracted debate. In addition, not having a cohesive plan for how to disburse QGIS funds results in funding being done in a very ad hoc manner – which in turn results in a skew of funding towards development related activities and away from other critical project activities such as improvement of user documentation, API documentation, sysadmin tasks and so on.
To get things done that volunteers don’t naturally gravitate towards doing, such as housekeeping, maintenance and so on.
To transparently spend QGIS.ORG funds to advance the QGIS project.
In this funding round, we are ring-fencing EUR 20,000 for the grant programme. We expect to run further grant calls in the future if this round proves to be a success and as funds allow.
Applicants may submit more than one proposal and the proposal may be on any topic that you think is relevant and beneficial to the greater QGIS community. Some examples of the kinds of topics you could propose are:
Updating and improving documentation
Updating and improving QGIS.org web infrastructure
Implementing a new feature in QGIS
Curating the pull request queue
Improving API documentation
Improving the API and help making QGIS 3.0 a reality
Rewriting and improving a part of the code base
A security review of QGIS
Helping new QGIS devs to get started with improved developer documentation and utilities
The closing date for applications is Thursday, 15 September 2016
PLEASE NOTE: All applications made here will be PUBLICLY VISIBLE, including your name.
Here are a list of frequently asked questions relating to the grant call. Please check back on this article regularly – we will update it as any new questions are raised so that everyone may benefit from the answers.
1) Q: Are collaborative proposals allowed?
A: One person should be the proposal lead though. Additional collaborators can be mentioned in the proposal details section.
2) Q: Can I make a proposal for a smaller amount?
3) Q: Can I make a proposal for a larger amount?
4) Q: Can I charge VAT / additional expenses on top of the grant allocation?
A: No, the amount should be all-inclusive.
5) Q: How will the grant awards be decided?
A: Grant applications will be decided on by vote of the QGIS Board Voting Members
6) Q: Can the grant be made on behalf of my company or a group of people?
A: Yes. Just note that any application you make should be inclusive of all costs, VAT, Taxes etc.
7) Q: How many grants will be awarded from the 20,000 Euros?
A: We expect to award at minimum two grants, possibly more if there are a number of smaller grant proposals that are worthwhile.
8) Q: Can I make more than one application?
9) Q: Is this like Google Summer of Code – a mentorship programme?
A: No. We will not provide mentorship – we expect that you are already an established developer or contributors to the QGIS project and do not need any ‘hand holding’ other than via normal community consultation processes like QEP’s.
10) Q: I am thinking of submitting a proposal to do XYZ. Would that be considered a valid proposition?
A: We don’t have any specific pre-conceived ideas of what a valid proposal is, so I would encourage you to make a submission if you think it is worthwhile. During the decision about which proposals to access, we will consider factors like:
how broadly useful the proposal is to all our users,
how unlikely is it that the feature or improvement would be done without Grant funding,
how much ‘value’ does the work bring to the project,
how feasible is it that the applicant will actually achieve their goals etc.
11) Q: Have you thought of how to handle situations where person A submits a proposal and, later, person B submits the same proposal but cheaper?
A: In these cases, we will use criteria such as the applicant’s standing in the community, the technical details of their implementation plan, etc. Price would probably be a low-weighted factor but certainly could enter into it if there is a significant difference.
12) Q: I’ve read that QGIS 3 might land in first quarter of 2017 (if everything goes well). Do you expect proposals to be tied to QGIS 3? Should bug fixes, plugins, PyQGIS book translations, should they be planned, developed, and tested against QGIS 3’s code base?
A: Where proposals relate to the code base, yes we would expect that they are ‘3.0 ready’ – though they do not necessarily have to be completed when 3.0 is released.
13) Q: Do you have an indication of how long it will take for the grants to be awarded after the closing date?
A: It’s a bit hard for me to say how long it is going to take. The process will entail asking the QGIS voting community to rank all the proposals. Depending on how many proposals we receive we will need to allow for sufficient time of this to happen. We hope we can do it within a month of the closing date for applications but it we get a hundred proposals we will need more time probably….
Q: I still have questions, who can I ask?
A: Please contact email@example.com if you have further questions, or write to the pic mailing list.
Time flies when you are having fun! It seems like only yesterday that I was writing about the 14th Hackfest in Gran Canaria. At the end of May 2016, we held the 15th QGIS hackfest! QGIS has been on an incredible journey since the project was started by Gary Sherman 14 years ago, and the fact that the hackfest was held in tandem with the 2nd QGIS International User conference is a testament to the growth and strength of the project.
2nd International QGIS Conference
Isn’t it amazing – we just held the second international QGIS User’s Conference! We really need to give credit to the amazing team who ran a totally seamless operation to organise the event: Gemma Pons, Toni Hernández, Josep Sitjar, Alexandre Busquets, Ferran Orduña, Rosa Olivella, Laura Olivas, & Lluís Vicens
We would like to give a special thank you to the University of Girona’s Director of GIS Service (SIGTE) – Gemma Boix who helped to organised the event as well as ensuring the institutional support for the event. In case I have missed mentioning someone by name, our thanks to all the other volunteers and the sponsors of the conference. Not only did the conference team host the conference event, they also covered a large part of the costs of the hackfest that followed the conference – for which we can’t thank them enough! OSGeo also supported the hack fest financially for which we are extremely grateful.
The conference team also extends their thanks to the attendees and presenters, instructors and developers who also actively participated on the event! For those interested in viewing the various talks at the conference, here are some handy links:
The QGIS.org project is in a very healthy state right now. For my talk at the user conference (video here) I got some fresh download stats from our servers and the numbers are quite astounding: QGIS 2.8 LTR (release Feb 2015) has been downloaded over 679, 000 times for the Windows Standalone installer (which includes all bug fix releases). Even after controlling for overestimates (from bot downloads, multiple downloads per user, “try and don’t use”) and underestimates (single downloads being distributed to many users) it is clear that we have an extremely large and constantly growing user base. By another metric, our most popular plugin, the OpenLayers plugin has been downloaded over 700 000 times!
Something else that is clear from the make up of the talks, workshop topics and attendees at the user conference and hackfest: QGIS is increasingly moving from single user environments to large multi-user deployments. This ‘edge of network effect’ is a common phenomenon in FOSS and is largely how Linux came to be such a lynchpin in the dev-ops world. Sys admins and (in the case of QGIS) GIS power users, test out the software on their own devices, see the potential for it in their workplace and start integrating it into their workflows in the office until eventually it has become a mission critical piece of software for an organisation.
One thing my slides probably do not make clear is that there is a huge amount of investment being made into QGIS and plugins for QGIS to fulfil a wide variety of needs. These investments are largely external to the project (i.e. not factored into the financial figures I mentioned in the talk) and happen in direct client-to-developer relationships completely bypassing (from a financial sense) the upstream QGIS.ORG project. This is a really good model since we do not need to deal with contract delivery, competing interests etc.
If we do a simple calculation based on direct QGIS.org revenue for 2015/2016 (around EUR 69, 000) to downloads, the average revenue per download of QGIS 2.8.x was around EUR 0.10. All of the money we receive into the project goes into improving the QGIS, maintaining infrastructure and funding travel and accommodation for hackfests. I mention these numbers both because they are interesting and because it is good to emphasise how incredibly grateful we are to each and every one of our sponsors and donors that support the project. We really do run on a shoe-string budget and we have audacious goals and a vision to put spatial decision making tools into the hands of everyone on the planet who wants to use them. Your sponsorship and donations are a key enabler to making this vision a reality!
By the way, my apologies for not mentioning Gary Sherman (our project founder) by name at the start of my talk – that was totally unintentional! Before I talk about the main activities, let me make a quick aside to mention the small excursion we took:
Underwater autonomous vehicles & QGIS
One of the really cool things we did at the hackfest was take a little side trip to visit the Computer Vision and Robotics Institute. Natália Hurtós (who works for the institute) kindly gave a bunch of us QGIS geeks a tour. The work they are doing building [relatively] cheap underwater autonomous vehicles is really awesome and inspiring – all the more so because Natalia is planning to build the mission planning tool using QGIS libraries!
Ok so what actually happened at the hackfest? Lets dig in and find out!
Cool stuff from the QGIS Hackfest
I am only going to focus on the hackfest here because the videos from the talks at the QGIS User Conference have been posted online (see above), so you can take in all the QGIS goodness you like from those. There was a lot going on at the hackfest, so these are only the nuggets I managed to cherry-pick from the talks.
Give processing some love
During the hackfest and user conference, Victor Olaya (lead developer for the QGIS processing framework) really did a great job of promoting the idea of writing processing plugins rather than ‘normal’ plugins. His argument is that most plugins that are intended to provide single purpose analytical capabilities (we are looking at you geeks about to write the 300th buffer plugin!) would be better off implemented as processing plugins:
the plugin author would not need to spend valuable time writing user interfaces, input validators and so on.
users of the plugins could chain the tool into complex workflows easily, rather than only being able to use it on a once off basis.
we would grow the amount of options available in the processing toolbox while at the same time reducing the sometimes overwhelming amount of choice in the QGIS Plugin Manager.
ILWIS processing tools coming soon
Also on the topic of plugins, Bas Restsios from the ILWIS Project gave a demo of the ILWIS software in order for QGIS developers to be more aware of its capabilities. Although currently Windows only, the ILWIS developers are in the process of porting the software to be based on the Qt5 framework, which means us Linux and OSX users will get to enjoy using it too soon. ILWIS is Open Source and does its rendering using OpenGL. Because of their smart rendering system, everything draws lightning fast. ILWIS packs in many remote sensing tools and raster analysis tools and should be on anyone’s radar if they are interested in FOSSGIS. The best part (from my point of view) is that Bas and his team members are also busy creating a set of processing plugins for QGIS that call out to ILWIS’s command line tools. This means you can expect a huge leap forward in the number of raster based analysis functions you can do with QGIS in the near future.
GeoPackage, JPEG2000, WFS improvements
It was really great to have Even Rouault (maintainer of GDAL/OGR) present at the hackfest. Even recently received core committer rights to the QGIS code repository and has been making great contributions by adding support for the OGC Geopackage format (death to shapefiles!). The shapefile format is long in the tooth and doesn’t serve the GIS community well as a de facto standard for GIS data interchange. It also doesn’t make a good format for intermediate representation of data processing outputs (e.g. when using multi algorithm processing models). Even has also been working on improvements to WFS support in QGIS which many will appreciate.
Not directly related to QGIS, but Even also mentioned he has been giving the GDAL driver for JPEG2000 some love – gaining good performance increases. This gives me some hope that there will be a viable open format alternative to ECW and MrSid in the future – something which QGIS will benefit from greatly. One of the interesting things that GDAL supports with the JPEG2000 specification is embedded vectors – you can write the raster with gdal_translate and then during creation pass a shapefile or GML stream to the gdal_translate command. I look forward to the day (not currently on the roadmap) where we can ship JP2 images with embedded vector masks and use those embedded vectors seamlessly in QGIS.
Planning for 3.0 release
We did some planning for the 3.0 release of QGIS and, in particular, fine tuned the plans for how we will manage the transition from QGIS 2.x to 3.x. In February 2016 I posted an outline of the general approach we planned to follow. Some developers felt that we would be better off having the core of the 3.0 transition work happening on the master branch of QGIS so that it has more attention and testing focussed on it. I guess there are two main interest groups to consider here, so I will break down the outcome between take-home points for developers and general users:
2.16 gets released off master.
After 2.16 we create a 2x branch and the 2x branch will be put into caretaker mode.
Support for Qt4 and Py2 will be discontinued in master and we do packaging only against Qt5 and Py3.
Once Qt5 and Py3 only are supported in master, nightly build packages may not be immediately available on master as we need to get the packaging systems update.
After 2.16 is released API breaking changes will be allowed in master (but code must build and packaging not broken, plugins will be broken).
All API breaking changes should be annotated by means of a Doxygen page patch to indicate what changes were made.
We target the API change window to end in Feb 2017 so that we can have a 3.0 release in March 2017.
We may do a 2.18 release off the 2x branch if there have been substantial changes in the 2x branch. If there are no substantial changes in the 2x branch following 2.16, we will not do a 2.18 release.
In January 2017 we will have a review to establish if all the API breaking changes are complete. If someone has specific implementation plans that are in progress and they need more time, we will (under agreement from the dev community) extend the API breaking window and maypush out the release date.
Following the release of 3.0 we will implement API freeze again and polish up the codebase for an LTR release based on 3.2
2.14 LTR support will be extended until 3.2 is ready
Note: for QGIS 3.x, a minimum of Qt 5.5 is recommended
Note: for QGIS 3.x a minimum of Python 3.4 is recommended
Note: We are looking for a volunteer to set up a windows build system using vagrant on win 7 so that we can test and build automaticallyl
For General Users
We propose that for general users, you rather share the following more easy to digest bullet list as it does not contain all the technical details above that mainly developers will care about.
2.18 release is not guaranteed
We are planning a 3.0 release in March 2017
This release date may be postponed – we will do a review in January to establish whether we are on track for the release date
The 3.0 release will break your plugins – we will publish a migration guide and tools to help you migrate your plugin to the new platform
If you have any queries about the process please feel free to contact us (via the developer or user mailing lists preferably)
Akbar Gumbira, our Google Summer of Code (GSOC) student joined us at the hackfest. For his GSOC project, Akbar is working on a unified way to share styles, symbols, ramps and markers in QGIS. The idea will be very similar to the QGIS plugin repository, where users can host their favourite cartographic elements for everyone else to enjoy. There was quite a lot of technical discussion about the exact mechanisms to use for hosting shared symbology with the key elements of the debate being about whether to use git as a hosting system, simple zip files or some other mechanism. If you wish to chat to Akbar about his work, visit this chat room or comment on the QEP.
OSX Packaging and Building
QGIS is an attractive proposition for OSX users since the big commercial vendors typically don’t support OSX. Since QGIS is built on cross platform technologies, this does not pose a huge limitation for us, but there are few OSX developers in the QGIS developer team and the platform requirements and constraints are pretty complex. Larry Shaffer has been doing awesome work to cut through the various issues and simplify the installation process – both for end users and for developers. It’s finally pretty easy to get a development environment up and running on a Mac now. Using brew (a package manager for OSX) you can install all the needed dependencies and get QGIS compiling in Qt Creator with full debugging support. The next challenge is going to be supporting this under Qt5 for QGIS 3.0 and Larry has been doing a bunch of great work to make that happen.
There are several web client frontends for QGIS out there. Lizmap is probably the most feature rich of them. Michaël Douchin (@kimaidou on twitter) showed off the latest version of Lizmap. If you are doing any web mapping, you really should check it out!
Marco Bernasocchi showed off the work they have been doing on QField – an Android app for your mobile device. The workflow for using QField is simple: Create a QGIS project on your desktop, copy it over to your device’s SD Card or internal storage, then open the same project on your device. Since it uses the same QGIS 2.14 backend as you are probably running on your desktop, all the cartographic elements from your desktop are supported – including the new 2.5D rendering. The main use case for QField is field data collection and new in QField is the ability to capture point data and edit / update feature attributes. This opens many possibilities for asset management and field survey work.
Marco shared some roadmap plans for QField including broader support for form elements. Value maps are already implemented, value relations, date and picture support are coming soon!. There is also beta support for digitising lines which should land in your installed version soon. I am eagerly looking forward to seeing how QField develops!
During the hackfest I did some testing of QFied using a PostGIS layer in a project with the data coming from a remote QGIS server – and it works (assuming you have internet connectivity). I also did some testing using BTSync to create a synchronised file system between my mobile device and my desktop. Edits to layers on the desktop and the mobile device can easily be pushed back and forth, making it very very easy to push out maps and new data to workers in the field.
There were lots of interesting things going on for those interested in testing. Alessandro Pasotti showed off some really cool stuff he has been working on for running python tests directly in QGIS instead of using a mock QGIS iface object. There are huge benefits to doing this since your tests run in a ‘real’ QGIS environment. He also showed off how they are testing QGIS plugins in Docker using the above mentioned technique.
If you do need / want to use an mock iface object, Matthias Kuhn has been promoting the use of the new qgis.testing python module improvements which includes an iface object which is comprehensive in terms of API stubs. For pythonistas:
from qgis.testing import start_app, unittest
Matthias has also created a unittest subclass that you can use which includes nice goodies like letting you do asserts that geometries match.
Matthias Kuhn, Nyall Dawson and others have really been leading the charge to build a more comprehensive test suite in QGIS and there were lots of other interesting tips and tricks been shown like how to make your travis tests run against multiple versions of QGIS – which can be very handy for plugin authors.
Victor Olaya also showed off the tester plugin – for automated GUI testing based on recorded interaction sequences.
Nice things for developers
Martin Dobias showed off some of the tools they have been developing including:
Martin also showed of the First Aid plugin which lets you capture tracebacks in a more elegant way and also debug your plugin, stepping through the code and execute python instructions in the current run context. The First AID plugin makes a great alternative to using remote debugging from an IDE which is time consuming to set up and technical for less experienced developers to do.
Victor Olaya showed of a plugin (the Lessons plugin) they are working on to facilitate interactive training in QGIS. It uses a simple dock interface to guide the user through a series of activities. Interestingly you can ask it to play recorded macros (saved as python code) of the active being explained so that you can see how it is done if you can’t figure it out yourself. Their authoring system also allows you to record these macros.
Docker is increasingly becoming a useful building block for those wishing to deploy QGIS in server side contexts. Patrick Valsecchi and Stéphane Brunner did some really awesome work refactoring the QGIS Server docker image – you can get a preview here https://github.com/pvalsecc/QGIS/tree/docker (read the README for usage notes). Patrick’s work strips down the size of the docker image to make it much less of an overhead to pull the image. We would like to eventually merge this into the upstream QGIS repo so that we have a versioned docker build set up for each QGIS release.
Martin Dobias showed off the Crayfish plugin. Crayfish includes some really awesome multi temporal visualisation tools – really useful if you have time slice data in NetCDF or similar formats and you want to view layers sequentially as animations. It also has cool symbology additions to show flow / directionality arrows over your raster.
GeoNode QGIS Server Demo
Etienne Trimaille demonstrated the work he, Ismail Sunni and Rizky Maulana have been doing to implement a QGIS Server backend for GeoNode. GeoNode is a spatial content management system that allows you to easily upload and share geospatial data on the web. With their implementation you can upload a shape file or TIFF along with a QGIS .qml style file and the layer is published directly as a tile service in the original QGIS styling. Bye-bye hard do create SLD’s, hello easy to create QGIS styles. (Disclaimer: I work with Etienne, Ismail and Rizky so I am tooting our own horn a bit here).
The work of the documentation team is often overlooked in the frenzy to enjoy the new cool stuff that developers churn out. Take a look over at the documentation GitHub pulse page to see how just as much effort is going on in the documentation work. During the hackfest Yves Jacolin and the rest of the documentation team were hard at work getting things ready for the next QGIS Manual release. If you are able to lend a hand with the documentation effort (no coding skills required), please do contact Otto Dassau, the QGIS Documentation Lead!
Here is also a quick-fire list of features that were shown off -that are coming down the QGIS conveyor belt – mostly for QGIS 2.16 but also new plugins and other efforts:
Style dock in QGIS 2.16 with undo / redo – Nathan Woodrow
Dynamic hillside rendering – Nathan Woodrow
Awesome new gradient editor – Nyall Dawson
D3 & Plotly charting with chart interaction showing on the layer – Matteo Ghetta and Michael Douchin
Forms improvements: the ‘drag and drop’ editor will support multiple columns – Matthias Kuhn
Table view will be able to filter which columns to show (to let you hide unwanted columns) – Matthias Kuhn
Action widget columns in the attribute table: you can place one or more action widgets (and icons) into the attribute table making it very easy to fire off an action (e.g. python script) for a particular record – Matthias Kuhn
Support for changing column order display in attribute table – Matthias Kuhn
In composer you can get have JSON of all layer relations that you can use in your html widget – Nyall Dawson
Aggregation in expressions. We will explain this more in the visual changeling for the next release, but you will be able to compute attribute aggregates (e.g. sum of areas) in your reports – Nyall Dawson
Default values for fields – Matthias Kuhn showed upcoming work to add support for field defaults in when adding a new record to a vector layer.
Transaction groups / cascaded editing mode when you are editing a feature so that related tables get put into edit mode (project property needs to be enabled) – again we will explain this more in the upcoming 2.16 changelog – Matthias Kuhn
SectorPlot plugin: a plugin to let you make ‘pizza plots’ of your data – Raymond Nijssen and Richard Duivenvoorde
Denis Rouzaud showed off work he has been doing to create a standard settings dialog that you can use in your plugins. It has support for different input types and layer chooser etc. They provide a settings dialog base class that you can derive from in your plugin which will take care of reading and writing your settings to Settings.
User profiles plugin – lets you create customised user interface for different user categories – Alex Bruy
Georeferenced PDFs and outputs from composer (sorry not yet GeoPDF) but you can use composer to create a PDF – and then add that into QGIS just like any other raster layer. Nyall Dawson
Nyall Dawson chatted about his work on QGIS Task manager – a tool to let you easily create concurrent background tasks and show the progress of each task in a simple UI. I believe he is looking for funding to help him get this finished, so if you are interested please contact him.
Bivariate legends plugin (not published yet). Thomas Gratier showed off work he is doing to support the creation of bivariate legends.
Albireo QGIS Fork
Sandro Mani showed off an awesome new QGIS front end they have been working on at Sourcepole. Their implementation creates a brand dew ribbon based UI paradigm for QGIS (don’t scoff, it’s actually pretty cool!). They have pared down the number of features and grouped functionality into discrete areas, both the menus and ribbon icons changing based on the context of what you are doing. You can see the source code here: https://github.com/sourcepole/kadas-albireo. Here are some bullet notes I made while watching his presentation:
3D Globe updates to get it ready for production use. They have also integrated the 3D extrusion support and lots of improvements to the globe that Matthias Kuhn did as part of his masters thesis. Its really exciting to think that these improvements may make their way into the official QGIS builds sometime soon and we can all enjoy them.
Redlining – you can sketch onto the map without saving those geometries in a specific layer.
Measuring & profile tools
Shift-drag for zoom – just like you can do in OpenLayers and Leaflet (Nyall then went and added this to QGIS for 2.16 – yay!)
Switch coordinate reference systems used for cursor position display easily from the status bar
There were many other cool features – some of them may make their way into the mainstream QGIS desktop. If you are looking to support users who need a simplified QGIS user interface, Albireo QGIS spin off is something to watch.
Girona was an absolutely awesome venue for the QGIS Conference and Hackfest. I hope the participants of the conference gained good benefit from the experience. The hackfest had around 50 participants and beyond the above notes, there was such a lot going on including work on the documentation system, discussions on the proposed community voter system (more on that in a follow up blog post) and many other things. I really encourage you to attend these events if you want to keep track of the leading edge of QGIS developers.
One thing we have been encountering lately in the QGIS project is plugin authors not understanding the licensing requirements for publishing their QGIS plugins. In this article I will try to clarify this a little:
QGIS is Open Source Software and provides a great platform for third parties to distribute additional functionality to users through our plugin system. QGIS is licensed under the GPL version 2 or greater. This license is provided with every copy of the QGIS and in the source code and is available on our web site here:
Under the terms of this license, it is a requirement that all plugins distributed via http://plugins.qgis.org (or through other repositories that may be self-hosted) should comply with the GPL version 2 or greater license. In particular all code included in any plugin should be made clearly and easily available in source form. It has come to our attention that some plugin authors are distributing plugins that do not comply with this condition.
We ask you to consider the fact that many thousands of hours of work and large amounts of financial outlay from individuals and companies has gone into the creation of QGIS. This work is done under the basis that in-kind contributions raise the quality and capabilities of the platform for everyone. When you create a plugin, you only need to spend a minimal amount of effort to solve your specific requirements because we have done the rest of the work needed to provide an entire platform for you and our community of users. This is a key value proposition of Open Source: ‘a rising tide floats all boats’.
By publishing the source code for your plugin, others may inspect the underlying code of your plugin and learn from it and use that knowledge to further improve the platform. Not releasing your source code breaks this model. Besides being a contravention of the licensing conditions under which you received the QGIS software, withholding your source code does not advance the body shared knowledge, and does not embrace the spirit of sharing that has made the QGIS project such a success to date.
Thus if you are a plugin author who is distributing your plugin without the accompanying source code, you need to be aware that the source code needs to be made available to each person who receives the binary sources for your plugin.
One query that plugin authors have raised is whether the requirement to publish the sources of their plugin precludes their ability to sell or otherwise commercially benefit from their plugin work. This is not the case – you can sell you plugin as long as you make the sources of your plugin available under the same license as QGIS to each purchaser.
Should you have any queries about how to better collaborate within the QGIS community we are available to you – please direct your queries to firstname.lastname@example.org
As many of you will be aware, in this last year we have been embarking on the process to transition from a loose-knit community organisation to a more formal organisation. This new organisation ‘QGIS.org’ will be a legal entity and will afford us a greater amount of flexibility in how we manage funds, legal agreements and so forth. You can read the statutes for the QGIS.org organisation for more info.
Under the statutes of the new organisation, there will be one user group voting member put forward for each registered user group (user group registration page is here). For each user group voting member we will have a community voting member elected. In addition there will be one voting member from the OSGEO leadership. It is probably best explained by way of a simple (contrived) example:
1) The Martian user group puts forward Joe Alien to be their QGIS User Group Voting Member.
2) QGIS committers put forward nominations for a matching Community Voting Member from within the community. Community member with the highest number of nominations is elected. Only people with git commit rights to an official QGIS repository or write access to the QGIS translation platform on transifex can put forward nominations. The nominated person can be any person from within the QGIS community who is willing to serve as a voting QGIS community member.
3) The Moon user group puts forward Janet Luna to be their QGIS User Group Voting Member.
4) QGIS committers again nominate a matching Community Voting Member
5) OSGEO puts forward one person to act as the OSGEO voting member.
Voting members will elect the QGIS Board, Board Chairman and approve budgets and the annual report. Once elected, the board will make day to day decisions on behalf of the project as needed. The OSGEO voting member will also serve to ensure that there are always an odd number of voting members to avoid deadlocks. Voting members will have their continued membership confirmed on a 3 yearly basis.
Since we are bootstrapping the QGIS.org organisation, we have not yet elected any voting members. Currently there are 10 QGIS User Groups registered (this number may be different when you read this due to new registrations), thus we would like to invite all people who have Git commit access to any official QGIS repo, or transifex write access to make their nominations for their community voting members. The 10 consenting nominees with the highest number of nominations will be appointed as community voting members.
We will close the nomination period on Wed 11 May if at least 10 unique persons have been nominated, otherwise as soon as 10 unique persons have been nominated.
We will maintain a list of the QGIS Voting Members on the web site for public viewing.
After all of the voting members have been elected they will be asked to elect the members of the board. Board members need not be voting members, any QGIS community member can be elected to the board. For formation of the new board, existing PSC members will automatically be nominated and the community voting members can also put up additional nominations. We would then put the election of the board members to a vote (maintaining the same number of members as are currently in the PSC for now). Once the board has officially been elected, the voting members can then select a chair for the board. Note that all voting will be done electronically via an online form.
By this process we will have a democratically elected board entrusted with stewardship of the new QGIS.org Organisation.
Each year we will hold an annual general meeting and voting members can elect a new chair, or re-elect the existing chair. Similarly on a rotational basis, board members will be up for re-election each year. If things are a little unclear above, I have made a simple diagram which hopefully clarifies things (see below).
Note that Gary Sherman (QGIS project founder) is board member emeritus and this will not be up for re-election as he has life long tenure on the board.
So if you are eligible to nominate voting members (i.e. you are a committer on any official QGIS repo or a Transifex author), please head over to the form provided to make your nomination!
Behind the scenes a lot has happened to get ready for Qt5 and Python3. On the same codebase that is becoming the next release QGIS 2.16. This is really a great thing since we can focus work on a single…
QGIS is a user friendly Open Source Geographic Information System that runs on Linux, Unix, Mac OSX, and Windows.
We are very pleased to announce the release of QGIS 2.14 ‘Essen’. Essen was the host city to our developer meet ups in October 2012 and 2014.
Long Term Release
This is a special release since it is designated an ‘LTR’ (Long Term Release). LTR releases will be supported with backported bug fixes for one year, and will be in permanent feature freeze (i.e. no new features will be added, only bug fixes and trivial updates). Note that we are in discussion to extend the term of our LTR releases to two years, but for technical reasons we will not do this until QGIS 3.2.
The purpose of LTR releases is to provide a stable and less frequently changing platform for enterprises and organizations that do not want to deal with updating user skills, training materials etc. more than once per year. The success of the LTR is very much down to you, our beloved users – we need your support to help funding bug fixes and making sure in your support contracts with support providers specify that any bug fixes done on your behalf are applied to the LTR branch as well as our normal development branch.
If an LTR is important to you, please consider also directly supporting the QGIS project, or encourage your commercial provider to use LTR as a basis for your enterprise solution so that everyone may benefit from a stable platform that is being continuously improved and refined. Note that for users and organizations that like to live on the frontier, our regular four monthly releases will continue unabated.
Note that 2.14 first enters the regular package repositories and will not immediately replace 2.8 in the LTR package repositories. That will happen when 2.16 is released.
Whenever new features are added to software they introduce the possibility of new bugs – if you encounter any problems with this release, please file a ticket on the QGIS Bug Tracker. (http://hub.qgis.org)
The source code and binaries for Windows, Debian and Ubuntu are already available via the large download link on our home page: http://qgis.org. More packages will follow as soon as the package maintainers finish their work. Please revisit the page if your platform is not available yet.
We would like to thank the developers, documenters, testers and all the many folks out there who volunteer their time and effort (or fund people to do so). From the QGIS community we hope you enjoy this release! If you wish to donate time, money or otherwise get involved in making QGIS more awesome, please wander along to qgis.org and lend a hand!
QGIS is supported by donors and sponsors. A current list of donors who have
made financial contributions large and small to the project can be seen on our
donors list. If you would like to become and official project sponsor, please
visit our sponsorship page for details.
Current Sponsors of QGIS:
Sponsoring QGIS helps us to fund our six monthly developer meetings, maintain project infrastructure and fund bug fixing efforts. A complete list of current sponsors is provided below – our very great thank you to all of our sponsors!
The Call for Papers and Workshops for the 2nd International QGIS User and Developer Conference, is still open!
Call for Presentations
Deadline: February 22nd
The QGIS Conference presentations are 20 minutes long, with time for Q&A at the end of each talk. Presentations may cover any aspect related with the use or development of QGIS software. Anyone can can submit a presentation proposal and take part in the conference as a presenter. The received proposals will be reviewed by the program committee.
There are two kinds of workshops depending on the duration: 2 hours or 4 hours. If you want to actively participate in the 2nd Int. QGIS Conference and impart a workshop, don’t hesitate to send your workshop proposal to email@example.com
The proposal should be a brief abstract pointing out the expected duration of the workshop (2 hours, 4 hours) as well as a few lines describing the content of the workshop, pre-requisites for the attendants (if needed), name of the instructor…
The instructors of the selected workshops will receive a free pass for the conference.
Ok so quick spoiler here: there is no QGIS 3.0 ready yet, nor will there be a QGIS 3.0 for some time. This article provides a bit more detail on the plans for QGIS 3.0. A few weeks ago I wrote about some of the considerations for the 3.0 release, so you may want to read that first before continuing with this article as I do not cover the same ground here.
A lot of consideration has gone into deciding what the approach will be for the development of QGIS 3.0. Unfortunately the first PSC vote regarding which proposal to follow was a split decision (4 for, 3 against, 1 abstention and 1 suggestion for an alternative in the discussion). During our PSC meeting this week we re-tabled the topic and eventually agreed on Jürgen Fischer’s proposal (Jürgen is a QGIS PSC Member and the QGIS Release Manager) by a much more unanimous margin of 8 for, 1 neutral and 1 absent. Jürgen’s proposal is largely similar to the Proposal 2 described in my previous posting. I want to make some special notes here about our discussion and subsequent decision which will hopefully help to clarify the thinking behind our decision for other interested observers. First let me lay out Jürgen’s plan in his own words:
“My preferred approach would still be:
Do a Qt5/PyQt5/Python3 branch in parallel, actually work on it until it’s ready, make it master and release it as 3.0
Meantime keep working on master (2.x) and keep releasing them every 4 months as usual
Everyone can work on the branch (s)he wants (or is hired to), but needs to consider if (s)he want to do it (or spend funds on):
only for 2.x: knowing that it will be released soon; but might become unusable because platforms drop support for stuff it depends on sooner or later
only for 3.x: not knowing when that will ever release or
for both: knowing that work needs to be done twice.
People adding features to the master branch will be responsible to ensure that their work gets merged to 3.0 branch.
As PSC we should maintain the environment for people to do something for QGIS – but we cannot tell them to – so we don’t have resources we can actually plan with and that means we can either release something when the big thing is ready or what we have in fixed intervals.” – Jürgen Fischer
What follows are some further details and clarifications to our preferred approach:
Why do parallel development?
Parallel development of 3.0 maintaining our master branch with 2.x code in it has advantages and disadvantages. First the advantages:
If we encounter major technical difficulties / release blockers in the 3.0 branch, it will not impact on our normal 3 monthly release cycle.
Our binary build systems (Linux, Windows and OSX binaries) will be unaffected until 3.0 is ready.
It is very likely that building 3.0 binaries on different platforms is going to have difficulties for each platform. For example OSGEO4W has no Python3 and Qt5 packages yet. Someone needs to see to the creation of the required package as a separate exercise from the actuals development of a version of QGIS that will take advantage of these updated libraries. We don’t yet know what problems may be in countered in preparing these.
“Don’t break what already works”: we have a working and relatively stable master branch and we don’t want to do a ‘cowboy stunt’ and break it. We prefer to wait until the 3.0 branch is mature, has passing tests and is known to work well before merging it into master and treating it as our ‘best we currently have’ master branch.
Of course nothing in life is completely easy, there are also some disadvantages:
Some developers may feel that running two mainstream branches is dilution of effort. To counter this, our public recommendation is that after 2.16 comes out, all QGIS contributors are strongly encouraged to provide their patches against the 3.0 branch. Any features applied to the master branch is not guaranteed to be part of the 3.0 release.
Regular merging of master to the 3.0 branch may prove more and more difficult over time as the two branches diverge more. Again we will strongly encourage that developers submitting new features after the 2.16 release do so against the 3.0 branch.
3.0 branch won’t have auto build system for nightly binaries in the beginning. Since we expect that the initial branch of 3.0 will break these anyway, Having a separate branch is actually an advantage here as it will give binary packages some time to get their build systems up to speed.
To clarify things for developers wondering where to commit their work: we discourage people from writing new features in master after 2.16 is released and rather ask them to make their changes in the 3.0 branch. Only those who really need to see their features in the next 2.18 release would have to dual commit.
Isn’t it better to work on 3.0 in the master branch?
Some queries have been raised about whether it would be better to rather work on 3.0 in the ‘master branch’ and relegate the 2.x code base to a side branch (instead of our intended approach which is to keep master tracking 2.x until 3.0 is ready and then merge it to master). We feel that keeping master tracking the 2.x code base is more conservative – it will not break existing packaging / build systems and if there is any major hitch in 3.0 development the release process will continue unabated based on the 2.x code set. While 3.0 is under development, package builders will have time to figure out the packaging process while still keeping the regular nightly builds against 2.x running. The implication of this is that 2.18 may contain only bug fixes which were applied to the 2.x branch and no significant new features.
The schedule will not be fixed
One thing that we want to make really clear (and was a key point in our many discussions) is that there will be no fixed release date for QGIS version 3.0. There are several reasons for this:
As a steering committee, we can only set the QGIS ship pointing in a given direction, our power to actually make work happen is extremely limited. This is because we are a community made up largely of volunteer developers or developers working on a commission basis for third party clients. We have no say in how these contributors spend their time.
We do not yet know which (if any) major technical issues will be encountered during the development of 3.0. Any such issues could very well delay the roll out of QGIS 3.0.
Instead our plan is to make the 2.16 release and then effectively move all developer effort to the 3.0 branch as best we can (through close liaison with our developer community).
To clarify things for those wondering when they will give 3.0 to their users: The actual release date for 3.0 its interterminate, but the general aim is still to try to encourage everyone to get it ready for 1 year from now. Remember that as an open source community we cannot directly ensure that project timelines are met since our developer force is largely volunteer based or work according to their own companies agendas.
Will 3.0 be a Long Term Release (LTR)?
It is our recommendation that we wait until 3.2 is ready before designating it an LTR – there are going to be large changes in the code base for 3.0 and we would rather stabilise things a bit before applying the LTR label to the release.
Looking forward to 3.0
Personally I am very much looking forward to the release of QGIS 3.0 – it represents another huge milestone in our project, it affords us a great opportunity to get rid of a lot of cruft out of our code base and API’s and it will arm us with a set of modern, new libraries that will see us through the next several years. Rock on QGIS 3.0!
Processing just got a new testing framework to improve the long-term stability of this important plugin. And you can help to improve it, even if you are not a software developer! This is yet another piece in our never-stopping crusade to…
From time to time, I read articles comparing ArcGIS vs QGIS. Since many of those articles are created from an ArcGIS user point of view, they invariably lead to biased observations on QGIS lack of features. It’s time for a QGIS user perspective, so bare with me on this (a bit) long, totally and openly biased post.
“Hello, my name is Alexandre, and I have been using… QGIS“
This is something I would say at an anonymous QGIS user therapy group. I’m willing to attend one of those because being recently and temporally forced to use ArcGIS Desktop again (don’t ask!), I really really really miss QGIS in many ways.
There was a time when I have used ArcGIS on the regular basis. I used it until version 9.3.1 and then decided to move away (toward the light) into QGIS (1.7.4 I think). At that time, I missed some (or even many?) ArcGIS features, but I was willing to accept it in exchange for the freedom of the Open Source philosophy. Since then, a lot have happened in the QGIS world, and I have been watching it closely. I would expect the same have happened in ArcGIS side, but, as far I can see, it didn’t.
I’m using top shelf ArcGIS Desktop Advanced and I’m struggling to do very basic tasks that simply are nonexistent in ArcGIS. So here’s my short list of QGIS functionalities that I’m longing for. For those of you that use ArcGIS exclusively, some of this features may catch you by surprise.
Warning: For those of you that use ArcGIS exclusively, some of this features may catch you by surprise.
“ArcGIS have transparency! It’s in the Display tab, in the layer’s properties dialog!”
Yes, but… you can only set transparency at the layer level. That is, either it’s all transparent, or it’s not…
In QGIS on the other end, you can set transparency at layer level, feature/symbol level, and color level. You can say that this is being overrated, but check the differences in the following images.
Notice that in QGIS you can set transparency at color level everywhere (or almost everywhere) there is a color to select. This includes annotations (like the ones in the image above), labels and composers items. You can even set transparency in colors by using the RGBA function in an expression! How sweet can that be?
This is one of QGIS’s pristine jewels. The ability to combine layers the way you would do in almost any design/photo editing software. At layer or at feature level, you can control how they will “interact” with other layers or features below. Besides the normal mode, QGIS offers 12 other blending modes: Lighten, Screen, Dodge, Darken, Multiply, Burn, Overlay, Soft light, Hard light, Difference, and Subtract. Check this page to know more about the math behind each one and this image for some examples
It’s not easy to grasp how can this be of any use for cartography before you try it yourself. I had the chance to play around while trying to answer this question.
A very common application for this functionality is when you want to add shadows to simulate the relief by putting a hill shade on top of other layers. In ArcGIS, you can only control the layer transparency, and the result is always a bit pale. But in QGIS, you can keep the strength of the original colors by using the multiply mode in the hill shade layer.
You can also use blending modes in the print composer items, allowing you to combine them with other items and textures. This gives you the opportunity to make more “artistic” things without the need to go post-processing in a design software.
Colour Picker Menu
Controlling color is a very important deal for a cartographer and QGIS allow you to control it like the professional you are. You can select your colours using many different interfaces. Interfaces that you might recognize from software like Inkscape, Photoshop, Gimp and others alike.
My favorite one is the color picker. Using color picker, you can pick colors from anywhere on your screen, either from QGIS itself or outside. This is quite handy and productive when you are trying to use a color from your map, it’s legend, a COLOURlovers palette or a company logo.
You can also copy/paste colors between dialogs, save and import color palettes, and you can even name a color and use it in a variable. With all this available for free, it’s hard to swallow Windows color selector😦.
Vector symbols renderers “powertools”
In ArcGIS, you have a few fancy ways to symbol your vector layers. You got: Single symbol, Unique values, Unique values many fields… and so on. At the first glance, you may think that QGIS lacks some of them. Believe me, it doesn’t! In fact, QGIS offers much more flexibility when you need to symbol your layers.
For starters, it allows you to use fields or expressions on any of the symbols renderers, while ArcGIS only allows the use of fields. Powered by hundreds of functions and the ability to create your owns in python, what you can do with the expression builder has no limits. This means, for instance, that you can combine, select, recalculate, normalize an infinite number of fields to create your own “values” (not to mention that you can tweak your values labels, making it ideal to create the legend).
And then, in QGIS, you have the special (and kinda very specific) renderers, that make you say “wooooh”. Like the Inverted polygons that allow you to fill the the outside of polygons (nice to mask other features), the Point displacement to show points that are too close to each others, and the Heatmap that will transform, ON-THE-FLY, all your points in a layer into a nice heatmap without the need to convert them to raster (and that will update as soon as you, or anyone else, edits the point features).
But I have left the best to the end. The “One rendered to Rule them all”, the Rule-based symbols. With the rule-based renderer, you can add several rules, group them in a tree structure, and set each one with a desired symbol. This gives QGIS users full control of their layer’s symbols, and, paired with expression builder and data-defined properties, opens the door to many wonderful applications.
One of my favorite (and missed) features in QGIS is the Map Composer’s Atlas. I know that ArcGIS has its own “Atlas”, the Data Driven Pages, but frankly, it’s simply not the same.
I believe you know the basic functionally that both software allow. You create a map layout and choose a vector layer as coverage, and it will create an output panned or zoomed for each of the layer’s feature. You can also add some labels that will change according to the layers attributes.
But in QGIS, things go a little bit further…
Basically, you can use coverage layer’s attributes and geometry anywhere you can use an expression. And, in QGIS, expressions are everywhere. That way, most layers and map composer items properties can be controlled by a single coverage layer.
So, if you pair Atlas it with QGIS data-defined properties, rule-based symbols and expressions, ArcGIS Data Driven Pages are no match. You don’t think so? Try to answer this question then.
Tip: If you really want to leverage your map production, using Spatialite or Postgis databases you can create the perfect atlas coverage layers from views that fit your needs. No data redundancy and they are always updated.
Label and Style dialogs
This one is more of a User Experience thing than an actual feature, but you won’t imagine how refreshing it is to have all Style and Labels options in two single dialogs (with several tabs, of course).
Using the symbol menu in ArcGIS makes me feel like if I’m in the Inception movie, at some point in time, I no longer know where the hell am I. For example, to apply a dashed outline in a fill symbol I needed to open 5 different dialogs, and then go back clicking OK, OK, OK, OK …
In QGIS, once in the properties menu, every setting is (almost) one click way. And you just need to press OK (or Apply ) once to see the result!
As an extra, you can copy/paste styles from one layer to another, making styling several layers even faster. And now you say:
“Don’t be silly! In ArcGIS you can import symbols from other layers too.”
Symbols? yes. Labels? No! And if you had lots of work setting your fancy labels, having to do the exact same/similar thing in another layer, it will make you wanna cry… I did.
(I think I will leave the multiple styles per layer vs data frames comparison for another time)
In a GIS world that, more and more, is evolving towards Open Data, Open Standards and OGC Web Services, this reveals a very mercantile approach by ESRI. If I were an ESRI customer, I would feel outraged. <sarcasm>Or maybe not… maybe I would thank the opportunity to yet invest some more money in it’s really advanced features…<\sarcasm>
In QGIS, like everything else, WFS is absolutely free (as in freedom, not free beer). All you need to do is add the WFS server’s URL, and you can add all the layers you want, with the absolute sure that they are as updated as you can get.
Fortunately for ArcGIS users with a low budget, they can easily make a request for a layer using the internet browser😛.
Or they can simply use QGIS to download it. But, in both cases, be aware that the layers won’t update themselves by magic.
I have already mentioned the use of expressions several times, but for those of you that do not know the expression Builder, I though I end my post with one of my all time favourite features in QGIS.
I do not know enough of ArcGIS expression builder to really criticize it. But, AFAIK, you can use it to create labels and to populate a field using the field calculator. I know that there are many functions that you can use (I have used just a few) but they are not visible to the common user (you probably need to consult the ArcGIS Desktop Help to know them all). And you can create your own functions in VBScript, Python, and JsScript.
On QGIS side, like I said before, the Expression Builder can be used almost everywhere, and this makes it very convenient for many different purposes. In terms of functions, you have hundreds of functions right there in the builder’s dialog, with the corresponding syntax help, and some examples. You also have the fields and values like in ArcGIS, and you even have a “recent expressions” group for re-using recent expressions with no the need to remember prior expression.
Besides, you can create your own functions using Python (no VBScript or JsScript). For this purpose, you have a separate tab with a function editor. The editor have code highlighting and save your functions in your user area, making it available for later use (even for other QGIS sessions).
These are certainly not the only QGIS features that I miss, and they are probably not the most limiting ones (for instance, not being able to fully work with Spatialite and Postgis databases will make, for sure, my life miserable in the near future), but they were the ones I noticed right away when I (re)open ArcGIS for the first time.
I also feel that, based on the QGIS current development momentum, with each QGIS Changelog, the list will grow very fast. And although I haven’t tested ArcGIS Pro, I don’t think ESRI will be able to keep the pace.
“Are there things I still miss from ArcGIS?” Sure. I miss CMYK color support while exporting maps, for instance. But not as much as I miss QGIS now. Besides, I know that those will be addressed sooner or later.
In the end, I kinda enjoyed the opportunity to go back to ArcGIS, as it reinforced the way I think about QGIS. It’s all about freedom! Not only the freedom to use the software (that I was already aware) but also the freedom to control software itself and it’s outputs. Maintaining the users friendliness for new users, a lot have been done to make power users life easier, and they feel very pleased with it (at least I am).
The Call for Presentations and Workshops for the 2nd International QGIS User and Developer Conference, is already open!
The Local Organising Committee of the 2nd International QGIS User and Developer Conference, is pleased to announce the call for Presentations and Workshops for the 2016 Conference to be held in Girona (May 25th – 26th). If you have an open source geospatial story to tell, based on QGIS, we want to hear it!
Call for Presentations Deadline: February 15th
The QGIS Conference presentations are 20 minutes long, with time for Q&A at the end of each talk. Presentations may cover any aspect related with the use or development of QGIS software. Anyone can can submit a presentation proposal and take part in the conference as a presenter. The received proposals will be reviewed by the program committee.
There are two kinds of workshops depending on the duration: 2 hours or 4 hours. If you want to actively participate in the 2nd Int. QGIS Conference and impart a workshop, don’t hesitate to send your workshop proposal to firstname.lastname@example.org
The proposal should be a brief abstract pointing out the expected duration of the workshop (2 hours, 4 hours) as well as a few lines describing the content of the workshop, pre-requisites for the attendants (if needed), name of the instructor…
The instructors of the selected workshops will receive a free pass for the conference.
Many of you out there may be wondering ‘when are we going to release QGIS 3.0?’. Last year (2015) we started investigating when and how we would release QGIS 3.0. We promised (see Anita Graser’s post about this) that we would convey clearly to our users and developers our plans well before making the QGIS 3.0 release. In this post I will try to lay out some of the considerations for a QGIS 3.0 release and at the end of this post there is an opportunity for you to present your ideas.
Typically (when following semantic versioning) a major release is reserved for times when you break the API of your software. Breaking API is not a trivial decision for the QGIS project since we have hundreds of thousands of users out there who depend on QGIS to ‘just work’, and many developers who need to maintain third party software written on top of the QGIS API.
From time to time breaking the API is necessary to accommodate updating the architecture with improved approaches, new base libraries and fixes to sub-optimal decisions made in the past.
What are the implications of breaking the API?
One of the reasons we are hesitant about releasing an API breaking QGIS 3.0 release is that it will have a huge impact, potentially breaking the hundreds of plugins in the plugin repository. Plugins would no longer compatible with new API and plugin authors would be required to manually review their plugins to identify and update the places in their plugins where the code is no longer compatible with the new API.
The breadth of the updates required depends largely on
how many backwards incompatible changes we make to the API
how many places plugin authors have used parts of the API which have changed
I will talk more about how we can mitigate API breaking changes further on in this article.
What will be the key changes for 3.0?
There are four key areas that we are looking to change in 3.0:
Updating Qt4 to Qt5: This is the basic set of libraries on which QGIS is built and provides a high level, platform independent abstraction layer for building a graphical user application. Qt also provides libraries for carrying out disk i/o, networking operations, and graphics drawing operations (key functionality for QGIS). Qt4 (on which QGIS is currently based) is now not actively being developed by the Qt library maintainers and we are expecting to experience problems in the near future building Qt4 for some platforms (e.g. OS X) or having readily accessible binaries (e.g. Debian Testing and the upcoming release of Debian “Stretch”). Making QGIS work with Qt5 has already been worked on (chiefly by Matthias Kuhn) who together with Marco Bernasocchi produce the Android “QField” port of QGIS which is based completely on Qt5. There are however some outstanding limitations in the newer Qt5 that impact on QGIS – in particular with the embedded web browser widgets (used chiefly in the QGIS composer but also a few other places in QGIS).
Updating PyQt4 to PyQt5: These are the python language bindings for Qt which the QGIS python API relies on. When we shift to the Qt5 C++ library, we also want to shift to the updated PyQt5 python library so that we can benefit from the new Qt5 API within the python environment too.
Updating Python 2.7 to Python 3: Currently we bundle in Python 2.7 in our windows installers and require 2.7 on other platforms where we do not co-bundle Python with QGIS. Python 3 is the latest version of python and is recommended by the Python project. Python 2 is slightly incompatible with Python 3 (in much the same way as QGIS 2 -> QGIS 3 will be incompatible). The python developers have made Python 3 largely backwards compatible to Python 2, but the compatibility in the opposite direction is not as good.
Improving the QGIS API itself: One of the issues with maintaining API compatibility between releases is that you have to live with your design choices for a long time. In QGIS we try our best not to break the API within a minor release series – not always with success as the more hard core developers will attest to. Releasing an API incompatible version of QGIS for 3.0 will give us an opportunity to ‘clean house’ by fixing things in the API that we are unhappy with. You can see a provisional list of proposed API changes for 3.0 by looking at the 3.0 API issues list.
Mitigating 3.0 API breakages
As I mentioned the 3.0 release will break API from the 2.x release of QGIS and there is the potential that many existing plugins, applications and other code that rely on the current API will be broken. So what can we do to mitigate the changes? Matthias Kuhn, Jürgen Fischer, Nyall Dawson, Martin Dobias and other core developers have been looking at ways to mitigate the number of API breaking changes whilst still advancing the QGIS codebase to be based on the next generation of libraries and its own internal API. During our last QGIS Project Steering Committee meeting we ran through various possibilities. Matthias Kuhn kindly joined the meeting to help clarify our options going forward which I have tried to summarise in the table below:
QGIS 2.14 LTR
QGIS 2.16 ???
4 months after 2.14
8 month cycle?
Update python code of core QGIS to be Python 3 compatible and PyQt5 compatible (partial implementation for key functionality e.g. console, python core plugins etc.)
Deprecated in Debian Stretch (due in a year)
Misses QWebView – new replacement not on all platforms. Also misses QPainter Engine.
Wrappers PyQt5 -> PyQt4 Provide ~90% backwards compatibility
There are two key things to note about Matthias’ proposal:
In the first phase, work would be done in the QGIS 2.x series to complete support for Qt5, PyQt5, Python 3.0 whilst still supporting Qt4, PyQt4 and Python 2.7. This implies that all changes made in the first phase would be backwards compatible with previous QGIS 2.x releases. Python wrappers will be introduced so that the old PyQt4 API can still mostly be used when compiling against Qt5, PyQt5, Python 3.0. When using QGIS compiled against Qt4, PyQt4 and Python 2.7 there would be no compatibility breakage.
In the second phase, we would work to produce QGIS 3.0 which introduces a number of API breaking changes, completely removing Python 2.7, Qt4 and PyQt4 support. The python wrappers produced in the first phase would be kept and relied on to ensure that a large proportion of python code (plugins, scripts etc.) developed for QGIS 2.x releases continue to work in QGIS 3.x releases. In this phase we would also introduce the QGIS API changes which may break some plugins. To address this we will provide a migration guide to try to ease the process for those moving from code depending on the QGIS 2.x releases to the QGIS 3.x releases.
There are a couple of ‘gotchas’ that we should raise at this point as the above makes the migration to QGIS 3.0 sound fairly painless.
The first thing we should emphasise is that while the approach laid out above tries to minimise the amount of work python script and plugins writers have to do, this will not be a 100% effort free solution for python coders using QGIS. There will very likely be cases where code needs to be adjusted and in all cases at the very least it will probably need to be reviewed in order to ensure that it still functions properly.
There is no formal funding set aside to pay for developers to spend their time working on the migration process. Because of this it is going to be incredibly hard to give accurate timelines as to how long each part of the process will take. We need to take this uncertainty into account in our planning. Of course we welcome donations to help make this happen.
There may be developers and institutions out there funding new features for the QGIS 2.x series and this may affect your work. You should include in your project plans and budgets some allocation to cope with the migration to the QGIS 3.x platform.
If we do the work in the ‘master branch’ there may be a protracted time during which our master branch is unstable and in flux due to ongoing updates towards QGIS 3.0.
If we do the work in a ‘3.0 branch’, we run the risk that the 3.0 development may drag on longer unless there is a devoted group of developers working on it and getting it ready to merge to master.
In the light of all the above information, we propose one of two courses of action:
Do an interim release of 2.16 and then commence work on 3.0 in master with an 8 month development window. Work on 3.0 related stuff could already begin in 2.16 (see python3/pytq5), only incompatible changes have to be postponed to post 2.16.
Advantage: Main focus of work would be in master branch. Work scheduled for the near future can be released in expected timelines. Plugins will continue to work with master. People can start to write and test their code in a portable manner.
Disadvantage: Difficult to determine timelines as we don’t have funding
Create a long running 3.0 branch for the port to Qt5, Python 3.0 and PyQt5 and call for developers to get their 3.0 work in there. Continue with 2.x releases with the usual frequency until 3.0 is ready.
Advantage: We can release it ‘when it’s ready’. If there is no funding for 3.0 work subsequent releases are not jeopardised.
Disadvantage: Duplication of effort as work in master coming in needs to be ported over to the 3.0 branch.
Do you have an alternative proposal? We would like to get all the proposals on the table so that we (the PSC in consultation with core developers) can make the best judgement of how to approach the nitty-gritty process of managing the QGIS 3.0 development process. If you wish to submit a proposal, please send it to me (email@example.com) with the subject line ‘QGIS 3.0 Proposal‘. Please keep your proposal very short and succinct as we just need the high level concepts.
Here is the proposal submitted by Matthias Kuhn that you can use as a reference of how we might like a proposal to look:
QGIS 2.16 Release as usual in 4 months
-> PyQt5 Support
-> Python 3 Support
-> Wrapper library for PyQt4/PyQt5
-> Maybe a helper transition script that does 80% of the rewrite
-> All old plugins still work
-> Some python code is updated (console, plugin manager, processing) to
have some guidelines and experience how to update python code
-> For future debian, mac osx… versions there’s a qt5 version around
(with almost no plugins working)
During the same time: make some noise that QGIS 3 is coming and we need
everybody to put some money and dev time aside for it and that it’s
going to be amazing.
After that: 8 months break for 3.0 (maybe some betas after 4 months and
every month after)
Back to normal, everybody happy – except the lazy plugin devs who didn’t
After getting QField up and running in Android 5, we felt it was time to start documenting how QField works, we started documenting how to install and use QField. We also added a section on how to handle your data…
Local governments on the municipality or provincial level often have to deal with DXF/DWG/DGN documents delivered from architects, urban planners or engineering companies. As an example, cadastral or utility offices or companies often have to import CAD documents to either check deliveries about correctness or import selected objects into their central database. Another requirement is being able to visualize planned objects, such as buildings or bridges alongside with the cadastral data.
QGIS lacks proper import of CAD data so far – DXF files can only be imported on very basic levels (without styling and labeling), DWG files can’t be imported at all.
A requirements document has been written to implement import of DXF/DWG (and potentially also DGN) files into QGIS, while maintaining styling, labeling, 3D, curves and blocks.
The company Norbit, with Jürgen Fischer, submitted an offer. Jürgen is a long-term QGIS developer and QGIS.ORG board member. Norbit has extensive experience with bridging GIS and CAD.
The plan is to use the Teigha library of the OpenDesign Alliance (ODA) to read the CAD documents. Either QGIS.ORG or OSGeo would become a member of the ODA, to get access to the source code and for distributing the Teigha libraries with QGIS binaries.
At this time we are looking for organisations or companies that help funding the effort. The offer from Norbit is over 32k €. We are looking for institutional crowd funders with minimum contributions of 1k Euros. If you are interested in contributing, please contact by mail at andreas (at) qgis (dot) org. We can send you an invoice for your contribution.
Addition: if you want to do smaller, targeted donations, please use the regular donation channels and mention “CAD import” in the payment instructions.
Last weekend (November 5th – November 8th, 2015) was a special occasion for the QGIS Project – we convened the 14th our ‘hackfest’ meeting!
QGIS hackfests are events where an organiser provides a space for collaboration and members of the community converge and self-organise to improve the QGIS project. We are very much dependent on the good will of our hosts – typically universities – for providing these spaces and the logistics needed to manage the provision of accommodation, food, internet and other needs of our participants over the duration of the hackfest. For this event our gracious hosts were Universitas de Las Palmas de Gran Canaria. Our presence at the University was endorsed and facilitated by:
Rector of ULPGC: José Regidor García
ULPGC Manager: Conrado Domínguez Trujillo
CIO of ULPGC and Director of OSL (“oficina de software libre”, or Free Software Office): Jose Pablo Suárez Rivero.
Councilor of New Technologies Area, Public Administration and Sports at Las Palmas de Gran Canaria Town Hall: Aridany Romero Vega
Senior professor of computer Science: Agustín Trujillo Pino
The local coordination team was headed up by Pablo Fernández Moniz (CTO and GIS Analyst). Pablo did an amazing job in pulling all the strands together making the event happen! He was assisted by an awesome team of helpers, and participants:
Design: Ramsés Cabello Developer (staff at the event): Jaisiel Santana Almedia
Developer (staff at the event): Alejandro Sánchez Medina
Developer (staff at the event): Chano OrtegaTrujillo
Developer (staff at the event): Moises Bonilla
Our heartfelt thanks to all of the people who worked so hard to make this event happen, and to the participants who travelled from far and wide to attend the event. These events are a significant factor in the success of the QGIS project – they allow our developers and contributors to meet face to face and delve deep into technical issues that would be impossible to do under the somewhat disconnected collaboration environment provided by the internet.
There were just under 30 people in attendance at the hackfest – here are just a few of the activities that they were busy with during the hackfest:
Qt5 / Python 3 proof of concept by Matthias Kuhn
Topological editing with GRASS plugin (Radim Blazek) – video (not from event)
Geoserver explorer plugin (Luigi Pirelli)
General discussion and planning for the improvement of contribution workflows (pull requests, tickets, QEPs) and infrastructure (Redmine)
Improvement of analysis tools (ftools, processing) and their documentation
General bug fixes (Jürgen Fischer)
Improvements to the QGIS.org website (Richard Duivenvoorde and Anita Graser)
Implementation of a GitHub webhook to insert a ‘[FEATURE] issue’ so that documenters know to document new features as they are added to QGIS (Richard Duivenvoorde and Raymond Nijssen)
Processing support for loading Oracle data via OGR (Giovanni Manghi)
Launched the Italian QGIS User Group, updates to the documentation of Lizmap web client, managed the Italian translation, the python plugin queue (Paolo Cavallini)
Many of the costs not covered by our hosts were covered from central QGIS.org funds including food and travel costs for a number of participants. These funds originate from our donors and sponsorships and are an invaluable resource for the project to allow us to facilitate these events and move the development of QGIS forward. We encourage you to donate to the QGIS or become a sponsor if you would like to foster further innovation and support the long terms goals of the project.
If you would like further information about what was covered in the event, and to get a feel of the general atmosphere of the event, check out some of these links:
The QGIS project is growing in size, number of contributors/contributions and code complexity. This introduces challenges for the project, especially for maintaining quality. Maintaining and improving quality is one of the main concerns of the QGIS.ORG board and some core developers.
Past and current QA efforts
Tim Sutton introduced a first unit test framework several years ago. But it wasn’t very visible then and passing the tests as a prerequisite to make changes to master wasn’t enforced. About a year ago, Matthias Kuhn introduced automated unit testing for the Linux builds – using Travis continuous integration testing. At our github page you can always see whether the master version builds fine and whether the tests are passing – see the green “Build passing” button at the beginning of the Readme.md file.
Since then, OS X automated building/testing was added. Pull requests (new contributions from developers) can be tested prior to integration into the master branch. Another effort was to use coverity code scans to detect memory leaks. Nyall Dawson and others did a lot of improvements/fixes due to this automated code scans.
There is still a lot to do regarding quality and automated testing. Because continuous integration tests were only introduced about a year ago, it means that still a lot of areas in the code base remain untested. Also, the current unit tests do not test GUI interactions. There is an ongoing discussion if critical classes should have unit tests enforced for any code changes. Finally, our bug queue at http://hub.qgis.org/projects/quantum-gis/issues is still quite long, with lots of bigger and minor issues.
Your financial support really matters!
This is where your donations and sponsorships come in. For the past 3-4 releases we were able to pay 2-4 developers who worked for several days concentrating on bug fixing. For QGIS 2.12 we had Nyall Dawson, Jürgen Fischer and Larry Shaffer working on bug fixing. We also financially support Giovanni Manghi for working on our bug tracker (e.g. classifying issues correctly, trying to reproduce the issue, ask bug reporters for more details).
Now – you may wonder why they didn’t fix all the “BLOCKER”s first and then continue on with “HIGH”, “NORMAL” “LOW” issues – and why a lot of unreported issues and issues with label “LOW” were also fixed? The answer is that it is often more efficient for the developers to concentrate on a certain part of the code – e.g. concentrating on geometry, labeling and editing issues, as Nyall did for this round of bug fixing. This means that he would not only fix issues labeled as “BLOCKER” or “HIGH” but also other bugs that are in the same code. Finally, not all of the “Blocker” and “High” bugs are reproducable or the issue may be much too hard/time consuming to fix.
Due to your financial support, the 3 developers were able to fix the following list of issues for the QGIS 2.12 release – many of the fixes also get back-ported to QGIS 2.8 LTS release:
Sweep of all changed dialogs, ensuring tab order is correct
UNREPORTED: Only save effect element if it is non-default (decreases size of qgs project files)
HIGH: Fix map rotation not considered for ellipse marker data defined rotation (#13367)
HIGH: Maintain order of recent expressions (#13461)
NORMAL: Make sure recent expression group is always listed last (#13462)
NORMAL: [diagrams] Fix initial value of transparency slider not set (#13434)
UNREPORTED: Fix potential crashes in renderer widgets
NORMAL: Fix legends are empty if presets used with filtered legend (#13300)
UNREPORTED: Fix crashes and inconsistent ui when atlas is set to a geometryless layer
BLOCKER: Fix diagrams are always shown, regardless of setting (#13501)
BLOCKER: Fix fill ring tool used with advanced digitising crashes QGIS (#13355)
NORMAL: Fix add ring/fill ring tool works on first polygon (#13069)
BLOCKER: Fix missing sip bindings for renderers (#13545)
BLOCKER: Fix crash in label property dialog (#13543)
NORMAL: Fix hardcoded border for raster legend items (#13540)
BLOCKER: Fix symbols drawn multiple times in rule based renderer if symbol (#13220)
BLOCKER: Use a model for node editor table, fixed hang when node tool used on large feature (#13541)
BLOCKER: Fix node tool duplicates nodes when topological editing and snap are both enabled (#13466)
NORMAL: Fix broken data defined SVG marker outline width (#13423)
HIGH: Scale svg marker outline width to match context (#11522)
HIGH: Allow coloring of svg markers and svg fills when used with graduated/categorised renderers (#11658)
HIGH: Fix svg outline widths are incorrectly scaled (#11522)
UNREPORTED: Fix snapping options dialog not correctly initialised when loading projects
UNREPORTED: Fix uninitialized variables in advanced digitizing dock which meant that sometimes advanced digitising tools would be activated unexpectedly
NORMAL: Fix curved labels ignore line orientation placement flag (#5778)
UNREPORTED: [console] Move run button earlier in console editor toolbar (prevents it being hidden in overflow menu on small screens)
UNREPORTED: Fix fill and outline color for svg markers sometimes enabled even though SVG file does not support parameters
UNREPORTED: Fix svg marker colors not correctly restored from project
UNREPORTED: If svg files with params do not have a default value set, then don’t reset the fill/border color and border width when changing svg marker/svg fill SVG files (made the behaviour consistent between the svg marker and the other marker symbols)
NORMAL: Fix svg symbols are shown in white and hard to see in svg picker (#10908)
NORMAL: Fix refining rule based renderer using expression (#10815)
UNREPORTED: Fix crash when changing symbol types on windows
BLOCKER: Fix split parts tool only leaves one of the newly created parts (#13421)
BLOCKER: Fix using add part tool to add part to geometryless rows (#12885, #11319)
UNREPORTED: Fix some potential crashes with edit tools and null geometry
UNREPORTED (thought I’d submitted this years ago but can’t find the issue now): Allow adding features with empty geometry via attribute table
HIGH: Allow delete part tool to remove geometry from single type point and line layers (#13258)
LOW: Fix overview canvas background color not set (#11157)
Add some unit tests for QgsWKBTypes
NORMAL: When adding ring to a geometry, add z or m dimensions to the ring geometry if required (#7400, #7401)
NORMAL: Also show features with modified geometry when “show edited and new features” filter is active in attribute dialog (#11684)
BLOCKER: Fix broken apply button in label config dialog (#13543)
BLOCKER: Fix area calculation when OTF active and no ellipsoid, add unit test (#13601)
UNREPORTED: Fix exporting geometry collections to WKT would result in invalid WKT
UNREPORTED: Fix unable to import WKT using MultiPoint(1 1,2 2,…) format
UNREPORTED: Fix GeometryCollection WKT to support collections with multi* children and GeometryCollection children (allowed by spec)
Add a bunch of unit tests to geometry
– UNREPORTED: Fix calculation of area and length of mixed geometry collections
UNREPORTED: Fix geometry casting in python bindings (missing MultiLineString and GeometryCollection casts)
UNREPORTED: Fix calculation of length/perimeter for geometry collections
UNREPORTED: when creating geometry from WKT, upgrade dimensionality of geometry if coordinates are 3/4 dimensional
UNREPORTED: match dimensionality of collections to child dimensionality
UNREPORTED: fix area of curves was non-zero if curve is closed
UNREPORTED: don’t consider m values when testing for curve closedness
During my professional and personal life, I have worked with much different software, with all kinds of licenses. Most of them would be proprietary, closed and / or commercial code. So why devote my time and learning “exclusively” to Open Source?
Without going into detail about the differences between open source software and free software, there are several reasons why FOSS (Free and Open Source Software) interest me.
The first is obviously freedom. Be free to use the software in any context and for any purpose, without being limited by the costs of software acquisition and / or the rules and conditions imposed by the manufacturer (as many said free (as beer) software do). That allows me to, for example, familiarize myself with its features without having to use piracy, or, as a freelancer worker, develop my work based on my technical capacities rather than my financial ones.
Second, the community and collaborative factor. The fact that open source software is built by user and for users, where the main goal is to enhance the software functionality (and not just raise the number of sales), and wherein each enhancement introduced by an individual or company is subsequently shared for the benefit of the whole community, avoiding duplication and “reinventing the wheel”. This is done in part through a lot of volunteer work and constant sharing of knowledge, either by the programmers or users. Thus, together, we all evolve at the same time as the software itself. In addition, everyone can contribute in some way, by producing code, writing and preparing supporting documentation, translating them into other languages or just by reporting bugs.
Finally, the “costs“. The adoption of open source software in enterprises (including the public ones), allow them to focus their investments in training of the human resources and the possible (and desirable) sponsoring of new features that are essential for their workflow, usually for a small portion of the values to spend on the acquisition of commercial software (that usually “forces” the purchase of features that may never be used).