QGIS Planet

QGIS Development with Plugin Builder and pb_tool

The Plugin Builder is a great tool for generating a working plugin project that you can customize.

One of the main tasks in the development cycle is deploying the plugin to the QGIS plugin directory for testing. Plugin Builder comes with a Makefile that can be used on Linux and OS X to aid in development. Depending on your configuration, the Makefile may work on Windows.

To help in managing development of your projects, we've come up with another option---a Python tool called pb_tool, which works anywhere QGIS runs.

Here's what it provides:

Usage: pb_tool [OPTIONS] COMMAND [ARGS]...

  Simple Python tool to compile and deploy a QGIS plugin. For help on a
  command use --help after the command: pb_tool deploy --help.

  pb_tool requires a configuration file (default: pb_tool.cfg) that declares
  the files and resources used in your plugin. Plugin Builder 2.6.0 creates
  a config file when you generate a new plugin template.

  See http://g-sherman.github.io/plugin_build_tool for for an example config
  file. You can also use the create command to generate a best-guess config
  file for an existing project, then tweak as needed.

Options:
  --help  Show this message and exit.

Commands:
  clean       Remove compiled resource and ui files
  clean_docs  Remove the built HTML help files from the...
  compile     Compile the resource and ui files
  create      Create a config file based on source files in...
  dclean      Remove the deployed plugin from the...
  deploy      Deploy the plugin to QGIS plugin directory...
  doc         Build HTML version of the help files using...
  list        List the contents of the configuration file
  translate   Build translations using lrelease.
  validate    Check the pb_tool.cfg file for mandatory...
  version     Return the version of pb_tool and exit
  zip         Package the plugin into a zip file suitable...

In the command summary, a description ending in ... means there is more to see using the help switch:

pb_tool zip --help
Usage: pb_tool zip [OPTIONS]

  Package the plugin into a zip file suitable for uploading to the QGIS
  plugin repository

Options:
  --config TEXT  Name of the config file to use if other than pb_tool.cfg
  --help         Show this message and exit.

The Configuration File

pb_tool relies on a configuration file to do its work. Here's a sample pb_tool.cfg file:

# Configuration file for plugin builder tool
# Sane defaults for your plugin generated by the Plugin Builder are
# already set below.
#
[plugin]
# Name of the plugin. This is the name of the directory that will
# be created in .qgis2/python/plugins
name: TestPlugin

[files]
# Python  files that should be deployed with the plugin
python_files: __init__.py test_plugin.py test_plugin_dialog.py

# The main dialog file that is loaded (not compiled)
main_dialog: test_plugin_dialog_base.ui

# Other ui files for dialogs you create (these will be compiled)
compiled_ui_files: foo.ui

# Resource file(s) that will be compiled
resource_files: resources.qrc

# Other files required for the plugin
extras: icon.png metadata.txt

# Other directories to be deployed with the plugin.
# These must be subdirectories under the plugin directory
extra_dirs:

# ISO code(s) for any locales (translations), separated by spaces.
# Corresponding .ts files must exist in the i18n directory
locales: af

[help]
# the built help directory that should be deployed with the plugin
dir: help/build/html
# the name of the directory to target in the deployed plugin
target: help

The configuration file is pretty much self-explanatory and represents that generated by Plugin Builder 2.6 for a new plugin. As you develop your code, you simply add the file names to the appropriate sections.

Plugin Builder 2.6 will be available the week of the QGIS 2.6 release. In the meantime, you can use pb_tool create to create a config file. See the pb_tool website for more information.

Deploying

Here's what a deployment looks like with pb_tool:

$ pb_tool deploy
Deploying will:
            * Remove your currently deployed version
            * Compile the ui and resource files
            * Build the help docs
            * Copy everything to your .qgis2/python/plugins directory

Proceed? [y/N]: y
Removing plugin from /Users/gsherman/.qgis2/python/plugins/TestPlugin
Deploying to /Users/gsherman/.qgis2/python/plugins/TestPlugin
Compiling to make sure install is clean
Skipping foo.ui (unchanged)
Compiled 0 UI files
Skipping resources.qrc (unchanged)
Compiled 0 resource files
Building the help documentation
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v1.2b1
loading pickled environment... done
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
no targets are out of date.

Build finished. The HTML pages are in build/html.
Copying __init__.py
Copying test_plugin.py
Copying test_plugin_dialog.py
Copying test_plugin_dialog_base.ui
Copying foo.py
Copying resources_rc.py
Copying icon.png
Copying metadata.txt
Copying help/build/html to /Users/gsherman/.qgis2/python/plugins/TestPlugin/help

Getting Started

For details on installing and using pb_tool, see: http://g-sherman.github.io/pluginbuildtool

PyQGIS Programmer's Guide Available

The preview release of the PyQGIS Programmer's Guide is now available for purchase from Locate Press.

The PyQGIS Programmer's Guide

The PyQGIS Programmer's Guide is now available in both paperback and PDF. A sample chapter is also available for download.

The book is fully compatible with the QGIS 2.x series of releases.

Plugin Builder 2.8

Plugin Builder 2.8 is now available. This is a minor update that adds:

  • Suggestion for setting up an issue tracker and creating a code repository
  • Suggestion for a home page
  • Tag selection from a list of current tags
  • Documentation update, including information about using pb_tool to compile, deploy, and package your plugin
  • New URLs for Plugin Builder's home page and bug tracking

Optional is now Recommended

In previous versions the following items were "Optional" when creating a new plugin:

  • Bug tracker
  • Home page
  • Repository
  • Tags

We've changed those from "Optional" to "Recommended" because they are important for the success and longevity of your plugin(s). Setting up a code repository on GitHub automatically gives you issue tracking and the ability for others to collaborate with fixes and enhancements through pull requests.

Using GitHub also gives you the ability to setup a home page right from your repository using GitHub pages.

Adding one or more tags to your plugin helps people find them easier when browsing the QGIS Plugins website.

Getting It

You can install Plugin Builder 2.8 from the Plugins -> Manage and Install Plugins... menu. Version 2.8 works on QGIS versions 2.0 and up.

QGIS Training Opportunities

We're planning a couple of training classes for March:

  • Introduction to QGIS
  • Extending QGIS with Python

Each is a one day class and we plan to run them back to back. If you are local or just want to come to Alaska in March for some spring skiing, northern lights viewing, or to experience the equinox, please hop over to GeoApt and let us know so we can plan accordingly.

Faking a Data Provider with Python

QGIS data providers are written in C++, however it is possible to simulate a data provider in Python using a memory layer and some code to interface with your data.

Why would you want to do this? Typically you should use the QGIS data providers, but here are some reasons why you may want to give it a go:

  • There is no QGIS data provider
  • The generic access available through OGR doesn't provide all the features you need
  • You have no desire to write a provider in C++
  • No one will write a C++ provider for you, for any amount of money

If you go this route you are essentially creating a bridge that connects QGIS and your data store, be it flat file, database, or some other binary format. If Python can "talk" to your data store, you can write a pseudo-provider.

To illustrate the concept, we'll create a provider for CSV files that allows you to create a layer and have full editing capabilities using QGIS and the Python csv module.

The provider will:

  • Create a memory layer from a CSV file
  • Create fields in the layer based on integer, float, or string values in the CSV
  • Write changes back to the CSV file
  • Require the CSV file to have an X and Y field
  • Support only Point geometries

We'll use the cities.shp file that comes with the PyQGIS Programmer's Guide to create a CSV using ogr2ogr:

ogr2ogr -lco "GEOMETRY=AS_XY" -f CSV cities.csv ~/Downloads/pyqgis_data/cities.shp

This gives us a CSV file with point geometry fields.

If you don't want to roll your own, you can download the cities.csv file here.

Here's the header (first row) and a couple of rows from the file:

X,Y,NAME,COUNTRY,POPULATION,CAPITAL
33.086040496826172,68.963546752929688,Murmansk,Russia,468000,N
40.646160125732422,64.520668029785156,Arkhangelsk,Russia,416000,N
30.453327178955078,59.951889038085938,Saint Petersburg,Russia,5825000,N

Signals and Slots

We'll look at the code in a moment, but here is the general flow and connection of signals from the layer to slots (methods) in our provider:

These are all the connections we need (signal -> slot) to implement editing of both geometries and attributes for the layer.

The Plugin Code

The plugin is pretty simple; when the tool is clicked, it opens a dialog allowing you to select a CSV file, then uses the CsvLayer class to read the file, create the memory layer, make connections (signals -> slots), and add it to the QGIS canvas.

The run Method

Here's the run method from the plugin (csv_provider.py):

 1 def run(self):
 2     """Run method that performs all the real work"""
 3     # show the dialog
 4     self.dlg.show()
 5     # Run the dialog event loop
 6     result = self.dlg.exec_()
 7     # See if OK was pressed
 8     if result:
 9         csv_path = self.dlg.lineEdit.text()
10         self.csvlayer = CsvLayer(csv_path)

The dialog (csv_provider_dialog.py) contains an intimidating warning and allows you to select a CSV file to load (lines 4-6).

Line 9 gets the path of the selected file from the dialog and line 10 creates the layer using the selected file. From that point on, you interact with the layer using the regular QGIS editing tools. We need to keep a reference to the layer (self.csvlayer), otherwise all our connections get garbage collected and we lose the "link" to the CSV file.


The CsvLayer Class

The CsvLayer class manages the creation, loading, and editing of the CSV file. First let's look at the methods in csv_layer.py that create the layer from our CSV file.

Creating the Layer from the CSV

The __init__ method examines the header and a sample row to determine the names and field types to be created in the layer. To read the CSV file, we use the csv module.

Here's the __init__ method:

 1 def __init__(self, csv_path):
 2     """ Initialize the layer by reading the CSV file, creating a memory
 3     layer, and adding records to it """
 4     # Save the path to the file so we can update it in response to edits
 5     self.csv_path = csv_path
 6     self.csv_file = open(csv_path, 'rb')
 7     self.reader = csv.reader(self.csv_file)
 8     self.header = self.reader.next()
 9     logger(str(self.header))
10     # Get sample
11     sample = self.reader.next()
12     self.field_sample = dict(zip(self.header, sample))
13     logger("sample %s" % str(self.field_sample))
14     field_name_types = {}
15     # create dict of fieldname:type
16     for key in self.field_sample.keys():
17         if self.field_sample[key].isdigit():
18             field_type = 'integer'
19         else:
20             try:
21                 float(self.field_sample[key])
22                 field_type = 'real'
23             except ValueError:
24                 field_type = 'string'
25         field_name_types[key] = field_type
26     logger(str(field_name_types))
27     # Build up the URI needed to create memory layer
28     self.uri = self.uri = "Point?crs=epsg:4326"
29     for fld in self.header:
30         self.uri += '&field={}:{}'.format(fld, field_name_types[fld])
31 
32     logger(self.uri)
33     # Create the layer
34     self.lyr = QgsVectorLayer(self.uri, 'cities.csv', 'memory')
35     self.add_records()
36     # done with the csv file
37     self.csv_file.close()
38 
39     # Make connections
40     self.lyr.editingStarted.connect(self.editing_started)
41     self.lyr.editingStopped.connect(self.editing_stopped)
42     self.lyr.committedAttributeValuesChanges.connect(self.attributes_changed)
43     self.lyr.committedFeaturesAdded.connect(self.features_added)
44     self.lyr.committedFeaturesRemoved.connect(self.features_removed)
45     self.lyr.geometryChanged.connect(self.geometry_changed)
46 
47     # Add the layer the map
48     QgsMapLayerRegistry.instance().addMapLayer(self.lyr)

Once the CSV file is opened, the header is read in line 8, and a sample of the first row is read in line 11.

Line 12 creates a dict that maps the field names from the header to the corresponding values in the sample row.

Line 16-25 look at each sample value and determine if its type: integer, real, or string.

Lines 28-30 create the URI needed to create the memory layer, which is done in line 34.

The add_records method is called to read the CSV and add the features in line 35.

Lastly we make the connections needed to support editing of the attribute table and the CSV file in response to the actions in our Signal/Slot diagram (lines 40-45).

Here is the add_records method that reads the CSV and creates a corresponding feature in the newly created memory layer:

 1 def add_records(self):
 2     """ Add records to the memory layer by reading the CSV file """
 3     # Return to beginning of csv file
 4     self.csv_file.seek(0)
 5     # Skip the header
 6     self.reader.next()
 7     self.lyr.startEditing()
 8 
 9     for row in self.reader:
10         flds = dict(zip(self.header, row))
11         # logger("This row: %s" % flds)
12         feature = QgsFeature()
13         geometry = QgsGeometry.fromPoint(
14             QgsPoint(float(flds['X']), float(flds['Y'])))
15 
16         feature.setGeometry(geometry)
17         feature.setAttributes(row)
18         self.lyr.addFeature(feature, True)
19     self.lyr.commitChanges()

Each row in the CSV file is read and both the attributes and geometry for the feature are created and added to the layer.

Managing Changes

With the layer loaded and the connections made, we can edit the CSV using the regular QGIS editing tools. Here is the code for the connections that make this happen:

 1 def editing_started(self):
 2     """ Connect to the edit buffer so we can capture geometry and attribute
 3     changes """
 4     self.lyr.editBuffer().committedAttributeValuesChanges.connect(
 5         self.attributes_changed)
 6 
 7 def editing_stopped(self):
 8     """ Update the CSV file if changes were committed """
 9     if self.dirty:
10         logger("Updating the CSV")
11         features = self.lyr.getFeatures()
12         tempfile = NamedTemporaryFile(mode='w', delete=False)
13         writer = csv.writer(tempfile, delimiter=',')
14         # write the header
15         writer.writerow(self.header)
16         for feature in features:
17             row = []
18             for fld in self.header:
19                 row.append(feature[feature.fieldNameIndex(fld)])
20             writer.writerow(row)
21 
22         tempfile.close()
23         shutil.move(tempfile.name, self.csv_path)
24 
25         self.dirty = False
26 
27 def attributes_changed(self, layer, changes):
28     """ Attribute values changed; set the dirty flag """
29     if not self.doing_attr_update:
30         logger("attributes changed")
31         self.dirty = True
32 
33 def features_added(self, layer, features):
34     """ Features added; update the X and Y attributes for each and set the
35     dirty flag
36     """
37     logger("features added")
38     for feature in features:
39         self.geometry_changed(feature.id(), feature.geometry())
40     self.dirty = True
41 
42 def features_removed(self, layer, feature_ids):
43     """ Features removed; set the dirty flag """
44     logger("features removed")
45     self.dirty = True
46 
47 def geometry_changed(self, fid, geom):
48     """ Geometry for a feature changed; update the X and Y attributes for
49     each """
50     feature = self.lyr.getFeatures(QgsFeatureRequest(fid)).next()
51     pt = geom.asPoint()
52     logger("Updating feature {} ({}) X and Y attributes to: {}".format(
53         fid, feature['NAME'], pt.toString()))
54     self.lyr.changeAttributeValue(fid, feature.fieldNameIndex('X'),
55                                   pt.x())
56     self.lyr.changeAttributeValue(fid, feature.fieldNameIndex('Y'),
57                                   pt.y())

The dirty flag is used to indicate that the CSV file needs to be updated. Since the format doesn't support random access to individual rows, we rewrite the entire file each time an update is needed. This happens in the editing_stopped method.

When attributes are changed and/or removed, the only action taken is to set the dirty flag to True.

When there is a geometry change for a feature, it is updated in the attribute table immediately to keep the X and Y values in sync with the feature's coordinates. This happens in the geometry_changed method.

You can view the full code for both the CsvLayer class and the plugin on GitHub or by installing the plugin. To install the plugin, you'll need to make sure you have the Show also experimental plugins checkbox ticked in the Plugin Manager settings. To help you find it, the plugin is listed as CSV Provider in the Plugin Manager.

Looking Further

A few things about this implementation:

  • It is an example, not a robust implementation
  • It lacks proper error handling
  • There is no help file
  • It could be extended to support other geometry types in CSV
  • In its current form, it may not work for other CSV files, depending on how they are formatted (take a look at the Add Delimited Text Layer dialog in QGIS to see what I mean)
  • If you want to enhance this for fun or profit---well for fun, fork the repository and give me some pull requests

A couple of final points:

  • Going this route can be a lot more work than using an existing data provider
  • This method can be (and has been) used successfully to interface with a data store for which there is no data provider

A Quick Guide to Getting Started with PyQGIS on Windows

Getting started with Python and QGIS can be a bit overwhelming. In this post we give you a quick start to get you up and running and maybe make your PyQGIS life a little easier.

There are likely many ways to setup a working PyQGIS development environment---this one works pretty well.

Contents

Requirements

  • OSGeo4W Advanced Install of QGIS
  • pip (for installing/managing Python packages)
  • pb_tool (cross-platform tool for compiling/deploying/distributing QGIS plugin)
  • A customized startup script to set the environment (pyqgis.cmd)
  • IDE (optional)
  • Vim (just kidding)

We'll start with the installs.

Installing

Almost everything we need can be installed using the OSGeo4W installer available on the QGIS website.

OSGeo4W

From the QGIS website, download the appropriate network installer (32 or 64 bit)

  • Run the installer and choose the Advanced Install option
  • Install from Internet
  • Choose a directory for the install---I prefer a path without spaces such as C:\OSGeo4W
  • Accept default for local package directory and Start menu name
  • Tweak network connection option if needed on the Select Your Internet Connection screen
  • Accept default download site location
  • From the Select packages screen, select the following for installation:
    • Desktop -> qgis: QGIS Desktop
    • Libs -> qt4-devel (needed for lrelease/translations)
    • Libs -> setuptools (needed for installing pip)

When you click Next a bunch of additional packages will be suggested---just accept them and continue the install.

Once complete you will have a functioning QGIS install along with the other parts we need. If you want to work with the nightly build of QGIS, choose Desktop -> qgis-dev instead.

If you've already installed QGIS using the OSGeo4W installer, just install the qt4-devel and setutools packages. If you installed QGIS using the standalone installer, the easiest option is to remove it and install from OSGeo4W. You can run both the standalone and OSGeo4W versions on the same machine, but you need to be extra careful not to mix up the environment.

Setting the Environment

To continue with the setup, we need to set the environment by creating a .cmd script. The following is adapted from several sources, and trimmed down to the minimum. Copy and paste it into a file named pyqgis.cmd and save it to a convenient location (like your HOME directory).

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W
call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%"\apps\grass\grass-6.4.3\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python;
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27\Lib\site-packages
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
set PATH=C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\Vim\vim74;%PATH%
cd %HOMEPATH%\development
cmd.exe

You should customize the set PATH statement to add any paths you want available when working from the command line. I added paths to my git and vim installs.

The cd %HOMEPATH%\development statement starts the shell in my normal working directory---customize or remove as you see fit.

The last line starts a cmd shell with the settings specified above it. We'll see an example of starting an IDE in a bit.

You can test to make sure all is well by double-clicking on our pyqgis.cmd script, then starting Python and attempting to import one of the QGIS modules:

C:\Users\gsherman\development>python
Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import qgis.core
>>>

If you don't get any complaints on import, things are looking good.

Python Packages

We need a couple of Python packages as well.

pip

There are several ways to install pip, but since we installed setuptools we can use easy_install:

easy_install pip

Make sure to issue this command from your customized shell (double-click on pyqgis.cmd to start it).

pb_tool

With pip installed we can use it to install pb_tool:

pip install pb_tool

More information on using pb_tool is available on the project website.

Working on the Command Line

Just double-click on your pyqgis.cmd script from the Explorer or a desktop shortcut to start a cmd shell. From here you can use Python interactively and also use pb_tool to compile and deploy your plugin for testing.

IDE Example

With slight modification, we can start our IDE with the proper settings to recognize the QGIS libraries:

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W
call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%"\apps\grass\grass-6.4.3\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python;
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27\Lib\site-packages
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
set PATH=C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\Vim\vim74;%PATH%
cd %HOMEPATH%\development
start "PyCharm aware of Quantum GIS" /B "C:\Program Files (x86)\JetBrains\PyCharm 3.4.1\bin\pycharm.exe" %*

We only changed the last line, adding the start statement with the path to the IDE (PyCharm). If you save this to something like pycharm.cmd, you can double-click on it to start PyCharm. The same method works for other IDEs, such as PyDev.

Within your IDE settings, point it to use the Python interpreter included with OSGeo4W---typically at: %OSGEO4W_ROOT%\bin\python.exe. This will make it pick up all the QGIS goodies needed for development, completion, and debugging. In my case OSGEO4W_ROOT is C:\OSGeo4W, so in the IDE, the path to the correct Python interpreter would be: C:\OSGeo4W\bin\python.exe.

Make sure you adjust the paths in your .cmd scripts to match your system and software locations.

Workflow

Here is an example of a workflow you can use once you're setup for development.

Creating a New Plugin

  1. Use the Plugin Builder plugin to create a starting point [1]
  2. Start your pyqgis.cmd shell
  3. Use pb_tool to compile and deploy the plugin (pb_tool deploy will do it all in one pass)
  4. Activate it in QGIS and test it out
  5. Add code, deploy, test, repeat

Working with Existing Plugin Code

The steps are basically the same was creating a new plugin, except we start by using pb_tool to create a new config file:

  1. Start your pyqgis.cmd shell
  2. Change to the directory containing your plugin code
  3. Use pb_tool create to create a config file
  4. Edit pb_tool.cfg to adjust/add things create may have missed
  5. Start at step 3 in Creating a New Plugin and press on

Troubleshooting

Assuming you have things properly installed, trouble usually stems from an incorrect environment.

  • Make sure QGIS runs and the Python console is available and working
  • Check all the paths in your pygis.cmd or your custom IDE cmd script
  • Make sure your IDE is using the Python interpreter that comes with OSGeo4W


[1] Plugin Builder 2.6 will support generation of a pb_tool config file

Plugin Builder 2.8.1

This minor update to the Plugin Builder allows you to choose where your plugin menu will be located.

Previously your menu was placed under the Plugins menu. At version 2.8.1 you can choose from the following main menu locations:

  • Plugins
  • Database
  • Raster
  • Vector
  • Web

Plugins is the default choice when you open Plugin Builder. The value you choose is also written to the category field in your metadata.txt file. When you view your plugin in the Plugin Manager, the value of category is displayed, aiding folks in finding the menu location.

You can install Plugin Builder 2.8.1 from the Plugins -> Manage and Install Plugins... menu. Version 2.8.1 works on QGIS versions 2.0 and up.

Why QGIS Class Names Start with Qgs

If you're a developer, or have looked at the QGIS source code, you've likely noticed that most C++ classes in the project start with Qgs.

Back before the dark ages of QGIS, Trolltech (now Digia) allowed you to reserve name prefixes for classes that used the Qt framework.

Shortly afterwards, I reserved the gs prefix for my use, resulting in class names that start with Qgs.

You might think this is based on some mangling of words like QGIS or perhaps GIS, but it was purely egocentric:

As far as I can tell, reservation of prefix names is no longer possible. For a view into what it was like back then, take a look at the Internet Archive.

Although the choice of prefix wasn't based on technology or discipline, years later it seems to fit...

PyQGIS Resources

Here is a short list of resources available when writing Python code in QGIS. If you know of others, please leave a comment.

Blogs/Websites

In alphabetical order:

Documentation

Example Code

  • Existing plugins can be a great learning tool
  • Code Snippets in the PyQGIS Cookbook

Plugins/Tools

  • Script Runner: Run scripts to automate QGIS tasks
  • Plugin Builder: Create a starter plugin that you can customize to complete your own plugin
  • Plugin Reloader: Allows you to reload a plugin from within QGIS
  • pb_tool: Tool to compile and deploy your plugins

Books

Plugin Builder 2.8

The Update

Plugin Builder 2.8 is now available. This is a minor update that adds:

  • Suggestion for setting up an issue tracker and creating a code repository
  • Suggestion for a home page
  • Tag selection from a list of current tags
  • Documentation update, including information about using pb_tool to compile, deploy, and package your plugin
  • New URLs for Plugin Builder’s home page and bug tracking

Optional is now Recommended

In previous versions the following items were “Optional” when creating a new plugin:

  • Bug tracker
  • Home page
  • Repository
  • Tags

We’ve changed those from “Optional” to “Recommended” because they are important for the success and longevity of your plugin(s). Setting up a code repository on GitHub automatically gives you issue tracking and the ability for others to collaborate with fixes and enhancements through pull requests.

Using GitHub also gives you the ability to setup a home page right from your repository using GitHub pages.

Adding one or more tags to your plugin helps people find them easier when browsing the QGIS Plugins website.

Getting It

You can install Plugin Builder 2.8 from the Plugins -> Manage and Install Plugins… menu. Version 2.8 works on QGIS versions 2.0 and up.

PyQGIS Resources

Here is a short list of resources available when writing Python code in QGIS. If you know of others, please leave a comment.

Blogs/Websites

In alphabetical order:

Documentation

Example Code

  • Existing plugins can be a great learning tool
  • Code Snippets in the PyQGIS Cookbook

Plugins/Tools

  • Script Runner: Run scripts to automate QGIS tasks
  • Plugin Builder: Create a starter plugin that you can customize to complete your own plugin
  • pb_tool: Tool to compile and deploy your plugins

Books

A Quick Guide to Getting Started with PyQGIS on Windows

Getting started with Python and QGIS can be a bit overwhelming. In this post we give you a quick start to get you up and running and maybe make your PyQGIS life a little easier.

There are likely many ways to setup a working PyQGIS development environment—this one works pretty well.

Contents

Requirements

  • OSGeo4W Advanced Install of QGIS
  • pip (for installing/managing Python packages)
  • pb_tool (cross-platform tool for compiling/deploying/distributing QGIS plugin)
  • A customized startup script to set the environment (pyqgis.cmd)
  • IDE (optional)
  • Vim (just kidding)

We’ll start with the installs.

Installing

Almost everything we need can be installed using the OSGeo4W installer available on the QGIS website.

OSGeo4W

From the QGIS website, download the appropriate network installer (32 or 64 bit)

  • Run the installer and choose the Advanced Install option
  • Install from Internet
  • Choose a directory for the install—I prefer a path without spaces such as C:\OSGeo4W
  • Accept default for local package directory and Start menu name
  • Tweak network connection option if needed on the Select Your Internet Connection screen
  • Accept default download site location
  • From the Select packages screen, select the following for installation:
    • Desktop -> qgis: QGIS Desktop
    • Libs -> qt4-devel (needed for lrelease/translations)
    • Libs -> setuptools (needed for installing pip)

When you click Next a bunch of additional packages will be suggested—just accept them and continue the install.

Once complete you will have a functioning QGIS install along with the other parts we need. If you want to work with the nightly build of QGIS, choose Desktop -> qgis-dev instead.

If you’ve already installed QGIS using the OSGeo4W installer, just install the qt4-devel and setutools packages. If you installed QGIS using the standalone installer, the easiest option is to remove it and install from OSGeo4W. You can run both the standalone and OSGeo4W versions on the same machine, but you need to be extra careful not to mix up the environment.

Setting the Environment

To continue with the setup, we need to set the environment by creating a .cmd script. The following is adapted from several sources, and trimmed down to the minimum. Copy and paste it into a file named pyqgis.cmd and save it to a convenient location (like your HOME directory).

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W
call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%"\apps\grass\grass-6.4.3\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python;
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27\Lib\site-packages
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
set PATH=C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\Vim\vim74;%PATH%
cd %HOMEPATH%\development
cmd.exe

You should customize the set PATH statement to add any paths you want available when working from the command line. I added paths to my git and vim installs.

The cd %HOMEPATH%\development statement starts the shell in my normal working directory—customize or remove as you see fit.

The last line starts a cmd shell with the settings specified above it. We’ll see an example of starting an IDE in a bit.

You can test to make sure all is well by double-clicking on our pyqgis.cmd script, then starting Python and attempting to import one of the QGIS modules:

C:\Users\gsherman\development>python
Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import qgis.core
>>>

If you don’t get any complaints on import, things are looking good.

Python Packages

We need a couple of Python packages as well.

pip

There are several ways to install pip, but since we installed setuptools we can use easy_install:

easy_install pip

Make sure to issue this command from your customized shell (double-click on pyqgis.cmd to start it).

pb_tool

With pip installed we can use it to install pb_tool:

pip install pb_tool

More information on using pb_tool is available on the project website.

Working on the Command Line

Just double-click on your pyqgis.cmd script from the Explorer or a desktop shortcut to start a cmd shell. From here you can use Python interactively and also use pb_tool to compile and deploy your plugin for testing.

IDE Example

With slight modification, we can start our IDE with the proper settings to recognize the QGIS libraries:

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W
call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%"\apps\grass\grass-6.4.3\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python;
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27\Lib\site-packages
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
set PATH=C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\Vim\vim74;%PATH%
cd %HOMEPATH%\development
start "PyCharm aware of Quantum GIS" /B "C:\Program Files (x86)\JetBrains\PyCharm 3.4.1\bin\pycharm.exe" %*

We only changed the last line, adding the start statement with the path to the IDE (PyCharm). If you save this to something like pycharm.cmd, you can double-click on it to start PyCharm. The same method works for other IDEs, such as PyDev.

Within your IDE settings, point it to use the Python interpreter included with OSGeo4W—typically at: %OSGEO4W_ROOT%\bin\python.exe. This will make it pick up all the QGIS goodies needed for development, completion, and debugging. In my case OSGEO4W_ROOT is C:\OSGeo4W, so in the IDE, the path to the correct Python interpreter would be: C:\OSGeo4W\bin\python.exe.

Make sure you adjust the paths in your .cmd scripts to match your system and software locations.

Workflow

Here is an example of a workflow you can use once you’re setup for development.

Creating a New Plugin

  1. Use the Plugin Builder plugin to create a starting point [1]
  2. Start your pyqgis.cmd shell
  3. Use pb_tool to compile and deploy the plugin (pb_tool deploy will do it all in one pass)
  4. Activate it in QGIS and test it out
  5. Add code, deploy, test, repeat

Working with Existing Plugin Code

The steps are basically the same was creating a new plugin, except we start by using pb_tool to create a new config file:

  1. Start your pyqgis.cmd shell
  2. Change to the directory containing your plugin code
  3. Use pb_tool create to create a config file
  4. Edit pb_tool.cfg to adjust/add things create may have missed
  5. Start at step 3 in Creating a New Plugin and press on

Troubleshooting

Assuming you have things properly installed, trouble usually stems from an incorrect environment.

  • Make sure QGIS runs and the Python console is available and working
  • Check all the paths in your pygis.cmd or your custom IDE cmd script
  • Make sure your IDE is using the Python interpreter that comes with OSGeo4W

[1] Plugin Builder 2.6 will support generation of a pb_tool config file

QGIS Development with Plugin Builder and pb_tool

The Plugin Builder is a great tool for generating a working plugin project that you can customize.

One of the main tasks in the development cycle is deploying the plugin to the QGIS plugin directory for testing. Plugin Builder comes with a Makefile that can be used on Linux and OS X to aid in development. Depending on your configuration, the Makefile may work on Windows.

To help in managing development of your projects, we’ve come up with another option—a Python tool called pb_tool, which works anywhere QGIS runs.

Here’s what it provides:

Usage: pb_tool [OPTIONS] COMMAND [ARGS]...

  Simple Python tool to compile and deploy a QGIS plugin. For help on a
  command use --help after the command: pb_tool deploy --help.

  pb_tool requires a configuration file (default: pb_tool.cfg) that declares
  the files and resources used in your plugin. Plugin Builder 2.6.0 creates
  a config file when you generate a new plugin template.

  See http://g-sherman.github.io/plugin_build_tool for for an example config
  file. You can also use the create command to generate a best-guess config
  file for an existing project, then tweak as needed.

Options:
  --help  Show this message and exit.

Commands:
  clean       Remove compiled resource and ui files
  clean_docs  Remove the built HTML help files from the...
  compile     Compile the resource and ui files
  create      Create a config file based on source files in...
  dclean      Remove the deployed plugin from the...
  deploy      Deploy the plugin to QGIS plugin directory...
  doc         Build HTML version of the help files using...
  list        List the contents of the configuration file
  translate   Build translations using lrelease.
  validate    Check the pb_tool.cfg file for mandatory...
  version     Return the version of pb_tool and exit
  zip         Package the plugin into a zip file suitable...

In the command summary, a description ending in … means there is more to see using the help switch:

1
2
3
4
5
6
7
8
9
pb_tool zip --help
Usage: pb_tool zip [OPTIONS]

  Package the plugin into a zip file suitable for uploading to the QGIS
  plugin repository

Options:
  --config TEXT  Name of the config file to use if other than pb_tool.cfg
  --help         Show this message and exit.

The Configuration File

pb_tool relies on a configuration file to do its work. Here’s a sample pb_tool.cfg file:

# Configuration file for plugin builder tool
# Sane defaults for your plugin generated by the Plugin Builder are
# already set below.
#
[plugin]
# Name of the plugin. This is the name of the directory that will
# be created in .qgis2/python/plugins
name: TestPlugin

[files]
# Python  files that should be deployed with the plugin
python_files: __init__.py test_plugin.py test_plugin_dialog.py

# The main dialog file that is loaded (not compiled)
main_dialog: test_plugin_dialog_base.ui

# Other ui files for dialogs you create (these will be compiled)
compiled_ui_files: foo.ui

# Resource file(s) that will be compiled
resource_files: resources.qrc

# Other files required for the plugin
extras: icon.png metadata.txt

# Other directories to be deployed with the plugin.
# These must be subdirectories under the plugin directory
extra_dirs:

# ISO code(s) for any locales (translations), separated by spaces.
# Corresponding .ts files must exist in the i18n directory
locales: af

[help]
# the built help directory that should be deployed with the plugin
dir: help/build/html
# the name of the directory to target in the deployed plugin
target: help

The configuration file is pretty much self-explanatory and represents that generated by Plugin Builder 2.6 for a new plugin. As you develop your code, you simply add the file names to the appropriate sections.

Plugin Builder 2.6 will be available the week of the QGIS 2.6 release. In the meantime, you can use pb_tool create to create a config file. See the pb_tool website for more information.

Deploying

Here’s what a deployment looks like with pb_tool:

$ pb_tool deploy
Deploying will:
            * Remove your currently deployed version
            * Compile the ui and resource files
            * Build the help docs
            * Copy everything to your .qgis2/python/plugins directory

Proceed? [y/N]: y
Removing plugin from /Users/gsherman/.qgis2/python/plugins/TestPlugin
Deploying to /Users/gsherman/.qgis2/python/plugins/TestPlugin
Compiling to make sure install is clean
Skipping foo.ui (unchanged)
Compiled 0 UI files
Skipping resources.qrc (unchanged)
Compiled 0 resource files
Building the help documentation
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v1.2b1
loading pickled environment... done
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
no targets are out of date.

Build finished. The HTML pages are in build/html.
Copying __init__.py
Copying test_plugin.py
Copying test_plugin_dialog.py
Copying test_plugin_dialog_base.ui
Copying foo.py
Copying resources_rc.py
Copying icon.png
Copying metadata.txt
Copying help/build/html to /Users/gsherman/.qgis2/python/plugins/TestPlugin/help

Getting Started

For details on installing and using pb_tool, see: http://g-sherman.github.io/plugin_build_tool

Why QGIS Class Names Start with Qgs

If you’re a developer, or have looked at the QGIS source code, you’ve likely noticed that most C++ classes in the project start with Qgs.

Back before the dark ages of QGIS, Trolltech (now Digia) allowed you to reserve name prefixes for classes that used the Qt framework.

Shortly afterwards, I reserved the gs prefix for my use, resulting in class names that start with Qgs.

You might think this is based on some mangling of words like QGIS or perhaps GIS, but it was purely egocentric:

/images/qgs_prefix.png

As far as I can tell, reservation of prefix names is no longer possible. For a view into what it was like back then, take a look at the Internet Archive:

http://web.archive.org/web/20030303075955/http://www.trolltech.com/documentation/naming.html

Although the choice of prefix wasn’t based on technology or discipline, years later it seems to fit…

The PyQGIS Programmer's Guide

The PyQGIS Programmer’s Guide is now available in both paperback and PDF. A sample chapter is also available for download.

The book is fully compatible with the QGIS 2.x series of releases.

PyQGIS Programmer's Guide Available

The preview release of the PyQGIS Programmer’s Guide is now available for purchase from Locate Press.

QGIS Training Opportunities

We’re planning a couple of training classes for March:

  • Introduction to QGIS
  • Extending QGIS with Python

Each is a one day class and we plan to run them back to back. If you are local or just want to come to Alaska in March for some spring skiing, northern lights viewing, or to experience the equinox, please hop over to GeoApt and let us know so we can plan accordingly.

Getting Paths with PyQGIS

When writing plugins or scripts it is often necessary to get information about the paths QGIS is using. For example, if we are writing a plugin that uses Python templates to create output based on user actions, we need to know the path to our installed plugin so we can find the templates. Fortunately the API provides an easy way to get at the information; here are a few examples:

  • QgsApplication.pluginPath(): location of core plugins
  • QgsApplication.prefixPath(): location where QGIS is installed
  • QgsApplication.qgisSettingsDirPath(): location of user settings
  • QgsApplication.iconsPath(): location of icons used in QGIS

We can get a summary of some paths used in QGIS using showSettings:

print QgsApplication.showSettings()
Application state:
QGIS_PREFIX_PATH env var:
Prefix:               /dev1/apps/qgis
Plugin Path:          /dev1/apps/qgis/lib/qgis/plugins
Package Data Path:    /dev1/apps/qgis/share/qgis
Active Theme Name:    default
Active Theme Path:    :/images/themes/default/
Default Theme Path:   :/images/themes/default/
SVG Search Paths:     /dev1/apps/qgis/share/qgis/svg/
                      /home/gsherman/.qgis2//svg/
User DB Path: /dev1/apps/qgis/share/qgis/resources/qgis.db

These paths are from my development install of QGIS on a Linux box; yours will be different.

The showSettings method is not really useful in a script or plugin since it returns a string containing line returns and tab characters. It’s mainly useful for debugging and development.

From within a plugin, you can get its path using

1
2
3
import os

plugin_path = os.path.dirname(os.path.realpath(__file__))

From the PyQGIS Programmers Guide

Welcome to New QGIS PSC Members

Nominations for the QGIS PSC closed at 00:00 UTC on August 25, 2013 [1].

With only one nominee for each role, the PSC unanimously moved to accept each without election.

The QGIS PSC welcomes new members Anita Graser, Richard Duivenvoorde, and Jürgen Fischer.

The PSC is now composed of:

  • Chair - Gary Sherman
  • Community Advisor - Otto Dassau
  • Design Advisor - Anita Graser
  • Financial and Marketing Advisor - Paolo Cavallini
  • Infrastructure Manager - Richard Duivenvoorde
  • Release Manager - Jürgen Fischer
  • Technical Advisor - Marco Hugentobler
  • Testing/QA Manager - Tim Sutton

The new PSC members begin their terms immediately.

[1] http://hub.qgis.org/projects/quantum-gis/wiki/Call_for_Nominations_August_2013

  • Page 1 of 4 ( 73 posts )
  • >>

Back to Top

Sponsors