QGIS Planet

Report back on the first QGIS User Conference in Nødebo, Denmark

I finally have some time to sit down and write up some thoughts on the QGIS User Conference and Developer Meeting (aka Hackfest) that we just held in Nødebo, Denmark. First up I need to thank Lene Fischer, who was the organiser and wowed us all with her relaxed and competent organisational approach to the conference. Thanks also to the University of Copenhagen School of Forestry – they sponsored the event by providing the venue and accommodation – and the venue was absolutely awesome with little cottages in the forest and all sorts of interesting diversions scattered around the forest. Lene gave me a list of names of people who helped to organise the event – I am sorry I have only got your first names but a very big thank you to you all!

 

Students: Runner, Shuttlebus, Kitchenaid, Cleaner, Info, Coordinator, Parking, Inn-keeper, Keyholder
Johanne
Thomas
Mikkel M
Steffen
Christian
Mikkel N
Ida
Louise
Anita
Thyge
Rune
Nanna
Peter
Jens
Heidi
Simon
Employees at University of CopenhagenCoordination, Accomodation, Bed&Linnen, Computer, Kitchen, Network, Tent/chairs/, Cookiebaker, Supporter, Cheerleader, Lifgt, Microphones/projector, Webpage, DTP,
Anne
Irene
Aleksander
Klaus
Vivian
Nicolas
Brian
Mike
Bo
Lene
Bent
Henning
Poul
Peter
MereteSusanne

 

 

 

On the first day of the user conference, I got to present a session on ‘the future of QGIS’ (video feed here and continued here) which held more as a town hall style meeting with a few themes (desktop, server, mobile etc.) I think the participants enjoyed the format and it was equally novel for the general user community (who got to have their questions answered directly by developers) and the developers (who got to see what real users look like).

The QGIS User Conference had many interesting talks (you can see the complete programme here – along with links to the video stream for each talk). For me the most interesting things happening at the meetup (both user conference and hackfest parts were:

  • the fact that we had our first ever general users conference (with around 150 attendees)
  • the geometry checking tools developed by Sandro Mani from Sourcepole
  • the huge amount of effort and thought being put into the processing framework – if you haven’t already tried out the QGIS processing tools, do go and try them!
  • The server side plugin framework that Alessandro Pasotti is working on – see his blog post here too http://www.itopen.it/qgis-developer-meeting-in-nodebo/
  • The amount of polish being applied to QGIS – there are probably less ‘gee whizz’ new features and a lot more fixes and improvements – just take a look at the incoming pull requests to get a flavour of the kind of activity going on.
  • The new geometry system by Marco Hugentobler (also from Sourcepole) which will support curves and 3D geometries (z / m). The graphical user interface for working with the new geometries won’t come until a later release, but 2.10 will get the underlying support added (along with shims to provide backwards compatibility to the old geometry classes).
  • There were some interesting talks on using QGIS in a server side / headless / command line context – again check out the talks and video streams in the programme to watch talks by Martin Dobias, Dražen Odobašić.
  • QGIS on mobile is coming – Matthias Kuhn showed off the current state of QField – the Android native interface for field work based on QGIS he has been working on. See his blog post here too for his take on the week. While the Android work shows lots of promise, there are still lots of problems to be resolved – for example missing support for ‘Lollipop’ devices. Please consider sponsoring Matthias’ efforts if you can.
  • There is a heap of interesting stuff coming down the pipeline from Nyall Dawson for the production of print maps and rendering effects for map renderers. Nyall also showed off some other very interesting ideas for context based variables that can be used in expressions – it’s hard to explain it in  a sentance or two – suffice to know that power users are going to have even more awesome tools at their fingertips for producing great maps.

One hot topic was ‘when will QGIS 3.0 be released’. The short answer to that question is that ‘we don’t know’ – Jürgen Fischer and Matthias Kuhn are still investigating our options and once they have had enough time to understand the implications of upgrading to Qt5, Python 3 etc. they will make some recommendations. I can tell you that we agreed to announce clearly and long in advance (e.g. 1 year) the roadmap to moving to QGIS 3.0 so that plugin builders and others who are using QGIS libraries for building third party apps will have enough time to be ready for the transition. At the moment it is still uncertain if there even is a pressing need to make the transition, so we are going to hang back and wait for Jürgen & Matthias’ feedback.

I apologise for not reporting on many of other interesting talks and birds of a feather meetings here – there was so much going on including work on documentation, translations, bug fixing, bug triaging that it is quite difficult to list it all here.

Two initiatives I was involved in at the meetup: the user certification programme and the formation of a QGIS legal entity. I am not going to post details here because things are not finalised yet (watch the mailing lists for details on the legal entity), but if you are interested in the certification programme, please get into contact – we have started drafting a roadmap for the roll out of our official curriculum. The QGIS project also got a huge boost from the QGIS Academy folks who will be contributing all their training resources right into the core of the QGIS documentation project (see Kurte Menke’s presentation notes on the programme).

We (Paolo Cavallini, Alessandre Pasotti, Nyall Dawson and myself) had a little roundtable discussion on the last day of the hackfest where we ran through some of the highlights from the week. You can listen to it here – or subscribe to the podcast at http://podcast.qgis.org (I will try to get back into the swing of making more regular episodes).

Well that wraps up my feedback for the event – I really encourage everyone to come along and join us on the next QGIS User Conference – it was fun, informal and informative!

How to run a Linux GUI application on OSX using Docker

Ok so here is the scenario:

You just got a nice new MacBook 15″ Retina computer thinking it would work as nicely for Linux as your 13″ MacBook did and then you discover that the hybrid Intel/Nvidia card support in Linux is a show stopper and the WebCam does not work under Linux.

Well that is what happened to me, so I decided to give working with OSX a try on this laptop with the help of docker for running all those essential apps that I use for development. One thing I was curious about was whether it would be possible to run native GUI (X11) applications from inside docker and have them show up on my OSX desktop. I turns out that it is fairly easy to do this – here is what I did:

Overview

  • Install brew
  • Install socat
  • Install XQuartz
  • Install Docker (I used Kitematic beta)
  • Grab a docker image that has a gui app you want to run (I used my the QGIS Desktop image published by Kartoza on the docker hub)
  • Run it forwarding the display to your OSX host

 

Digging In

Ok first install brew (an apt-like package manager for OSX).

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Now install socat – a command line tool that lets you redirect sockets in unix like OS’s – thankfully it runs in OSX too as it is a really neat tool!

brew install socat

Next we are going to install XQuartz – which basically gives you an X11 display client on your OSX desktop. Just grab the package at http://xquartz.macosforge.org/landing/ and do the usual OSX procedure for installing it.

Unfortunately docker does not run natively on OSX, and the whole boot2docker setup is probably quite difficult to explain to people. However there is a very nice (currently beta) docker client being developed for OSX called kinematic. I installed kinematic and then simply hit shift-command-t in order to get a bash shell with docker available in it.

Now grab my QGIS desktop image for docker:

docker pull kartoza/qgis-desktop

Once the image is downloaded we are done with the basic setup and can kick over to running our Linux GUI application (obviously QGIS in this example).

Running QGIS

Ok so there are four steps we need to do to run our Linux app:

  1. Start socat (in my testing it had to be done first)
  2. Start XQuartz
  3. Start Kinematic
  4. Start QGIS

I started socat like this:

socat TCP-LISTEN:6000,reuseaddr,fork UNIX-CLIENT:\"$DISPLAY\"

It will run in the foreground waiting for connections and then pass them over to XQuartz.

Next I started XQuartz (you can close the XTerm window that opens by default).  In X11 preferences in XQuartz, in the security tab, check both boxes:

Screen Shot 2015-04-13 at 23.40.21

Next I started kinematic, and pressed SHIFT-COMMAND-T to open a docker terminal.

Screen Shot 2015-04-13 at 23.16.21

Lastly I ran the QGIS docker container like this:

docker run --rm -e DISPLAY=192.168.0.3:0 \
    -i -t -v /Users/timlinux:/home/timlinux \
    kartoza/qgis-desktop qgis

You can mix in any standard docker options there – in this case I created  shared volume between my OSX home directory and a /home/timlinux directory in the container. You need to determine the IP address of your OSX machine and use it instead of the IP address listed after DISPLAY in the above command. Here is a nice picture of QGIS (from a Linux container) running on my OSX desktop:

Screen Shot 2015-04-13 at 23.52.21

 

 

This same technique should work nicely with any other GUI application under Linux – I will mostly use if for running tests of QGIS based plugins and for using QGIS in my docker orchestrated environments.

Generating an XYZ ASCII file from a QGIS raster

Someone wrote to me asking if it would be possible to generate an XYZ ASCII file from a single band raster layer in QGIS. No doubt there are more efficient ways (this approach is pretty slow but it works), but I thought it would be fun to show how you can iterate over a raster, writing out the value of each cell into a text file (along with the centroid coordinates for that cell).

 

 

 

 

To use the script, you should save it to your local machine, then open the python console and load the script in the python editor. Next select a single band raster and then run the script in the editor. If your raster is quite large, it will take some time to run. I have spent zero time trying to optimise the script – if someone has an idea for doing it faster, send me a patch and I will update the example above.

The generated output dataset will look something like this:

 

Longitude,Latitude,VI
8.31259406548,7.86128343221,10
8.31264849753,7.86128343221,16
8.31270292958,7.86128343221,18
8.31248520138,7.8613378416,15
8.31253963343,7.8613378416,17
8.31259406548,7.8613378416,24
8.31264849753,7.8613378416,46
8.31270292958,7.8613378416,47
...

The resulting script can be used with programs like gdal_grid or loaded back into QGIS as a vector layer using the Delimited Text provider:

 

QGIS 74ac7f0 - raster_to_xyz_171

How to create a QGIS PDF report with a few lines of python

Sometimes you want to automatically generate a report to reflect the latest state of your data. For example you may be capturing spatial data into a PostGIS database and want a snapshot of that every few hours expressed as a pdf report. This example shows you how you can quickly generate a pdf based on a QGIS project (.qgs file) and a QGIS template (.qpt file).

Using this approach you can generate all kinds of useful outputs without ever needing to open QGIS each time you generate the report. Simply create the needed project and template files and then run it like this:

python generate_pdf.py

QGIS course at AfQUA

Kartoza has just wrapped up a three-day Introduction to QGIS course that we gave pro bono at The African Quaternary: Environments, Ecology and Humans conference in Cape Town (AfQUA2015). Thank you to Kelly Kirsten from the local organising committee at the University of Cape Town for inviting us. Admire Nyakudya presented to about fifteen conference delegates from around the world. Attendees ran QGIS 2.6.1 on their own notebooks on the operating system of their choice. Some already use QGIS, some were experiencing QGIS for the first time and some some wanted to compare it with ArcGIS. The course content was guided largely by the QGIS training manual but as we usually do, adapted for the requirements, interests and pace of the class.

 

Kartoza QGIS class at AfQUA 2015

Admire presenting QGIS workshop at AfQUA

How to load a QGIS project in python

Today in a project we are working on we wanted to load a QGIS project. It takes  surprisingly few lines of code to make a small standalone application that loads a project and then shows it as a map in a window like this:

QGIS project in a standalone app

 

Here is the code I wrote to produce this:

The main bit of magic is the QgsLayerTreeMapCanvasBridge class which will convert your project into a layer tree so that the layers appear in the canvas. If you ever need to make a standalone python application with a nice map in it, consider using the QGIS API to do it!

A quick hack to select all multipart features in QGIS

Today we wanted to find all the multipart features in this layer:

The layer that needs multipart features identified

Here is a quick hack I did in QGIS to find them:

Just make sure that a vector layer is active in the layer list, paste the above code into the python console and all multipart features will be highlighted for you.

After running the above script, multipart features are hightlighted.

Setting up a Fedora 21 QGIS Workstation

I have been a long time Ubuntu user (I have actually been using it since Ubuntu 4.10 ‘Warty Warthog’) – the first official release. The advent of Ubuntu saw an end to my distro hopping whilst looking for the ‘perfect linux distro’.

Recently though, Ubuntu has been losing momentum in my opinion – especially in terms of supporting the latest Gnome desktop editions and catering for those of us who like to use a leading edge platform for our developer workstations. I was particularly curious to see if QGIS runs nicely under Wayland, the next-generation graphics environment for Linux.

I have been using docker heavily for the last year and have come to the point where I feel that the underlying Linux flavour is less important since I can fairly arbitrarily deploy applications in docker containers using whichever flavour of Linux inside the container is most convenient.

Thus I decided to try and see how easy it would be to get Fedora 21 installed on my MacBook 13″ Laptop which was running Ubuntu quite nicely until now. In the Gist below, I detail the various installation steps I took to get my standard suite of applications installed. These include:

  • docker
  • QGIS compilation build chain
  • PyCharm 4
  • Shutter
  • Skype
  • QtCreator / QtDesigner etc.
  • btsync
  • Google Chrome
  • vlc and assorted video codecs
  • keepassx
  • Elegance gnome theme (must-have if you use Gnome!)

 

Workspace 1_130

I will keep the above Gist updated as I tweak my configuration, but by and large the migration to Fedora has been fairly painless and I am enjoying working on the latest Gnome desktop. I was able to replicate pretty much all of the application stack I ran on Ubuntu, though in some cases the setup & installation of applications was a little more complex than on Ubuntu, and in one case (btsync-gtk-gui) I have not yet found a binary installation package.

How to build and debug QGIS with QtCreator

Here is how I build QGIS with QtCreator under ubuntu 14.04

Gary Sherman wins the Sol Katz award

This year Gary Sherman won the annual Sol Katz award. To quote the OSGEO page about the award:

The Sol Katz Award for Geospatial Free and Open Source Software (GFOSS) is awarded annually by OSGeo to individuals who have demonstrated leadership in the GFOSS community. Recipients of the award will have contributed significantly through their activities to advance open source ideals in the geospatial realm. The hope is that the award will both acknowledge the work of community members, and pay tribute to one of its founders, for years to come.

A couple of years ago I did an interview with Gary about the beginning of QGIS:

and part2:

I am so pleased that the Sol Katz award committee has seen fit to present this award to Gary. The fact that Gary created QGIS and open sourced its code has had a profound effect on many people’s lives – especially my own. Participating in QGIS for the last decade+ has immeasurably improved my own life and enabled me to eke out a comfortable living doing something I love. Hardly a day goes without me receiving an email in my inbox from someone in a far flung place telling me how they are using QGIS and how it is making their world a better place.

Presenting this award to Gary is a long overdue recognition and ‘thank you’ to the person who started a project that has become the rally point for an incredible, diverse and  friendly bunch of people, and a daily use tool for hundreds of thousands of users how there who may not even know who Gary is. QGIS rocks Gary, and so do you!

Here is Gary’s acceptance speech:

 

 

WMS Legend Plugin for Leaflet

This weekend I was updating our map gallery at http://maps.kartoza.com and I wanted to have WMS legends in my maps. The maps are mostly generated using QGIS server which also produces a nice looking graphic for its getLegendGraphic requests. Since Leaflet does not seem have a legend control out of the box, I wrote a small leaflet plugin to do it.

Leaflet WMS Legend Plugin

In the future I may extend the control to automatically fetch getLegendGraphics from all loaded WMS layers, but for now it simply takes a complete legend graphic URI as parameter.

Leaflet is a great web mapping client and extending it with little plugins is very easy to do. If you want to use the plugin I wrote, head over to the plugin repository and give it a try!

How to quickly transform a bounding box from one CRS to another using QGIS

Today I needed to convert a bounding box for a tilemill project that I want to bring into QGIS as a tile layer (more on that in a future post if I get it to work…).  I needed to convert a bounding box from EPSG:4326 (‘Geographic’) coordinates to EPSG:3857 (Spherical Mercator). Fortunately it is a fairly trivial process if you don’t mind writing a few lines of python in the QGIS python console:

box = QgsRectangle(-19.6875,-37.9962,59.0625,37.4400)
source_crs = QgsCoordinateReferenceSystem(4326)
dest_crs = QgsCoordinateReferenceSystem(3857)
transform = QgsCoordinateTransform(source_crs, dest_crs)
new_box = transform.transformBoundingBox(box)
new_box.toString()
u'-2191602.4749925746582448,-4578889.0142234507948160 : 6574807.4249777207151055,4500615.8633687794208527'

 

It really is quite trivial to do arbitrary once-off things in QGIS if you roll up your sleeves and get to grips with the python API!

Running QGIS desktop in a docker container

I love using docker – I have been tracking and learning docker since soon after it was announced and believe it is going to be a real game changer. I’ve been playing around with the different things one can do in a docker container and of course it is only natural that a ‘QGIS guy’ such as myself would start to think about using docker with QGIS. QGIS server in a docker container seems like a natural fit, but how about QGIS Desktop? Last night Richard Duivenvoorde and I were sitting around drinking tea and we thought we would give it a quick go – in fact it only took about half an hour to get something working….

 

Demo

 

Why?

I guess the first think you may ask is “why would you want to put QGIS desktop in a docker container?”. Well there are actually quite a few good reasons – here is a quick brain dump of reasons why you might want to run QGIS in a docker container:

  • Application sandboxing – keeping QGIS in a docker container means that you can keep it away from your other applications and data and frugally let it use only the resources you choose to. This is a general principle that can apply to any application you run on your desktop.
  • Capitalise on Ubuntu packages on a different host – if you are running CentOS or Arch or some other architecture, you may want to take advantage of the Ubuntu and Ubuntugis packages without trading out your entire OS. Now you can!
  • Running multiple versions of QGIS side by side – I already do this by using some little bash scripts that set paths and do magic before starting QGIS. Docker provides an alternative approach to this where each QGIS version can be in it’s own container.
  • Running different QGIS profiles – Perhaps you want to set up a profile where you have plugins x,y,z available and another where you have plugins a,b,c enabled – you could just create different docker images and launch a container based on the one that you want.
  • Known good deployment  – Setting up a linux with all the little bits and pieces needed to fully use QGIS takes some work and is vulnerable to breaking if you upgrade your OS. If you keep all that work in a docker image, the image will be unaffected by changes on the host system and you can do focussed updates to the image as needed. You can also do this one, publish the container and easily push it to your users in an enterprise environment.
  • Sharing a well integrated QGIS package – I have no love for Windows, but I must say that windows users have it good with the OSGEO4W and standalone installers for QGIS. With docker we could do something similar, where we create a well configured docker image and share it for the world to use…no more fiddling about trying to get stuff to run, just get the latest docker build and run QGIS with confidence knowing everything is set up for you.
  • Testing stuff – Testing is nice when you do it with a clean environment. With docker you can destroy and recreate the container each time you run it, reverting it to a clean state each time.

There are probably a bunch of other good reasons to play with this approach, but the above may be enough to get you curious and play…

Before I show you how to set things up, I should mention there are also some possible downsides:

  • Extra complications – adding docker into your mix is one thing you need to learn and understand – although the approach I show here requires only the most rudimentary understanding.
  • Statelessness – the statelessness of the container needs some extra steps to deal with. e.g. if you install a plugin and then shut down QGIS, when you start it again, it will be gone. Fear note docker volumes allow us to add state.
  • Overhead – Some may argue that running QGIS in a docker container is going to add overhead, making QGIS slower to run. Honestly in my testing I could not notice any difference.

Setting up

Before you can start you need to do a bit of setup and also note that my scripts provided make a few assumptions – you may wish to edit them to meet your needs. First you need to have docker installed on your OS. Under Ubuntu 14.04 you can simply do:

sudo apt-get install docker.io

Next you need to have my Dockerfile ‘recipe’ for building the docker image. It is available on our github repository (patches and improvements welcome!)

sudo apt-get install git
git clone git://github.com/kartoza/docker-qgis-desktop

Now go into the cloned repository and build the image:

cd docker-qgis-desktop
 ./build.sh

Its going to take a little while to build. After it is done, you should have:

  • a new docker image called kartoza/qgis-desktop in your docker images list
  • a launching script in your ~/bin directory
  • a line added to you ~/.bashrc that adds ~/bin to your path
  • A .desktop shortcut file added to ~/.local/share/application

Now reload your desktop (e.g. log out and in again) and look in your applications menu. You should find a new entry called ‘QGIS 2.in Docker’. Click on it and QGIS should launch.

What happens when I click that icon?

When you click the icon, a little script runs that starts up a new docker container, mounts your home directory into it and starts QGIS, sending its windows back to your deskop. Its all pretty seamless and feels like you are just running a locally installed application.

There are still some gotchas here since this is the first version of our script:

  • QGIS runs as root in the docker container which means its probably going to screw up the file permissions of any new file you create.
  • To get the QGIS window to display on your desktop I am using xhost + which some folks might not like
  • Your home folder is mounted into the container, but you won’t be able to see other files elsewhere on the host operating system

What’s next?

Currently we have QGIS 2.4 running in the container. We are going to work on providing the most polished installation possible inside the container. That means adding support for OrpheoToolBox (already added), SAGA, GRASS, MMQGIS, MrSid, ECW, ESRI FGDB etc. etc. that the docker container works ‘batteries included’ out of the box. If you would like to contribute, please consider forking our repo and submitting a pull request!

 

  • Page 1 of 1 ( 13 posts )

Back to Top

Sponsors