Page 1 of 1 (18 posts)

  • talks about »
  • pyqgis

Tags

Last update:
Thu Sep 3 12:05:31 2015

A Django site.

QGIS Planet

Open source IDF router for QGIS

This is a follow-up on my previous post introducing an Open source IDF parser for QGIS. Today’s post takes the code further and adds routing functionality for foot, bike, and car routes including oneway streets and turn restrictions.

You can find the script in my QGIS-resources repository on Github. It creates an IDFRouter object based on an IDF file which you can use to compute routes.

The following screenshot shows an example car route in Vienna which gets quite complex due to driving restrictions. The dark blue line is computed by my script on GIP data while the light blue line is the route from OpenRouteService.org (via the OSM route plugin) on OSM data. Minor route geometry differences are due to slight differences in the network link geometries.

Screenshot 2015-08-01 16.29.57


Open source IDF parser for QGIS

IDF is the data format used by Austrian authorities to publish the official open government street graph. It’s basically a text file describing network nodes, links, and permissions for different modes of transport.

Since, to my knowledge, there hasn’t been any open source IDF parser available so far, I’ve started to write my own using PyQGIS. You can find the script which is meant to be run in the QGIS Python console in my Github QGIS-resources repo.

I haven’t implemented all details yet but it successfully parses nodes and links from the two example IDF files that have been published so far as can be seen in the following screenshot which shows the Klagenfurt example data:

Screenshot 2015-07-23 16.23.25

If you are interested in advancing this project, just get in touch here or on Github.


A interactive command bar for QGIS

Something that has been on my mind for a long time is a interactive command interface for QGIS.  Something that you can easily open, run simple commands, and is interactive to ask for arguments when they are needed.

After using the command interface in Emacs for a little bit over the weekend – you can almost hear the Boos! from heavy Vim users :) – I thought this is something I must have in QGIS as well.  I’m sure it can’t be that hard to add.

So here it is.  A interactive command interface for QGIS.

commandbar

commandbar2

The command bar plugin (find it in the plugin installer) adds a simple interactive command bar to QGIS. Commands are defined as Python code and may take arguments.

Here is an example function:

@command.command("Name")
def load_project(name):
    """
    Load a project from the set project paths
    """
    _name = name
    name += ".qgs"
    for path in project_paths:
        for root, dirs, files in os.walk(path):
            if name in files:
                path = os.path.join(root, name)
                iface.addProject(path)
                return
    iface.addProject(_name)

All functions are interactive and if not all arguments are given when called it will prompt for each one.

Here is an example of calling the point-at function with no args. It will ask for the x and then the y

pointat

Here is calling point-at with all the args

pointatfunc

Functions can be called in the command bar like so:

my-function arg1 arg2 arg2

The command bar will split the line based on space and the first argument is always the function name, the rest are arguments passed to the function. You will also note that it will convert _ to - which is easier to type and looks nicer.

The command bar also has auto complete for defined functions – and tooltips once I get that to work correctly.

You can use CTRL + ; (CTRL + Semicolon), or CTRL + ,, to open and close the command bar.

What is a command interface without auto complete

autocomplete

Use Enter to select the item in the list.

How about a function to hide all the dock panels. Sure why not.

@command.command()
def hide_docks():
    docks = iface.mainWindow().findChildren(QDockWidget)
    for dock in docks:
        dock.setVisible(False)

alias command

You can also alias a function by calling the alias function in the command bar.

The alias command format is alias {name} {function} {args}

Here is an example of predefining the x for point-at as mypoint

-> alias mypoint point-at 100

point-at is a built in function that creates a point at x y however we can alias it so that it will be pre-called with the x argument set. Now when we call mypoint we only have to pass the y each time.

-> mypoint
(point-at) What is the Y?: 200

You can even alias the alias command – because why the heck not :)

-> alias a alias
a mypoint 100

a is now the shortcut hand for alias

WHY U NO USE PYTHON CONSOLE

The Python console is fine and dandy but we are not going for a full programming language here, that isn’t the point. The point is easy to use commands.

You could have a function called point_at in Python that would be

point_at(123,1331)

Handling incomplete functions is a lot harder because of the Python parser. In the end it’s easier and better IMO to just make a simple DSL for this and get all the power of a DSL then try and fit into Python.

It should also be noted that the commands defined in the plugin can still be called like normal Python functions because there is no magic there. The command bar is just a DSL wrapper around them.

Notes

This is still a bit of an experiment for me so things might change or things might not work as full expected just yet.

Check out the projects readme for more info on things that need to be done, open to suggestions and pull requests.

Also see the docs page for more in depth information


Filed under: Open Source, python, qgis Tagged: plugin, pyqgis, qgis

A Quick Guide to Getting Started with PyQGIS on Windows

Getting started with Python and QGIS can be a bit overwhelming. In this post we give you a quick start to get you up and running and maybe make your PyQGIS life a little easier.

There are likely many ways to setup a working PyQGIS development environment---this one works pretty well.

Contents

Requirements

  • OSGeo4W Advanced Install of QGIS
  • pip (for installing/managing Python packages)
  • pb_tool (cross-platform tool for compiling/deploying/distributing QGIS plugin)
  • A customized startup script to set the environment (pyqgis.cmd)
  • IDE (optional)
  • Vim (just kidding)

We'll start with the installs.

Installing

Almost everything we need can be installed using the OSGeo4W installer available on the QGIS website.

OSGeo4W

From the QGIS website, download the appropriate network installer (32 or 64 bit)

  • Run the installer and choose the Advanced Install option
  • Install from Internet
  • Choose a directory for the install---I prefer a path without spaces such as C:\OSGeo4W
  • Accept default for local package directory and Start menu name
  • Tweak network connection option if needed on the Select Your Internet Connection screen
  • Accept default download site location
  • From the Select packages screen, select the following for installation:
    • Desktop -> qgis: QGIS Desktop
    • Libs -> qt4-devel (needed for lrelease/translations)
    • Libs -> setuptools (needed for installing pip)

When you click Next a bunch of additional packages will be suggested---just accept them and continue the install.

Once complete you will have a functioning QGIS install along with the other parts we need. If you want to work with the nightly build of QGIS, choose Desktop -> qgis-dev instead.

If you've already installed QGIS using the OSGeo4W installer, just install the qt4-devel and setutools packages. If you installed QGIS using the standalone installer, the easiest option is to remove it and install from OSGeo4W. You can run both the standalone and OSGeo4W versions on the same machine, but you need to be extra careful not to mix up the environment.

Setting the Environment

To continue with the setup, we need to set the environment by creating a .cmd script. The following is adapted from several sources, and trimmed down to the minimum. Copy and paste it into a file named pyqgis.cmd and save it to a convenient location (like your HOME directory).

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W
call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%"\apps\grass\grass-6.4.3\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python;
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27\Lib\site-packages
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
set PATH=C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\Vim\vim74;%PATH%
cd %HOMEPATH%\development
cmd.exe

You should customize the set PATH statement to add any paths you want available when working from the command line. I added paths to my git and vim installs.

The cd %HOMEPATH%\development statement starts the shell in my normal working directory---customize or remove as you see fit.

The last line starts a cmd shell with the settings specified above it. We'll see an example of starting an IDE in a bit.

You can test to make sure all is well by double-clicking on our pyqgis.cmd script, then starting Python and attempting to import one of the QGIS modules:

C:\Users\gsherman\development>python
Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import qgis.core
>>>

If you don't get any complaints on import, things are looking good.

Python Packages

We need a couple of Python packages as well.

pip

There are several ways to install pip, but since we installed setuptools we can use easy_install:

easy_install pip

Make sure to issue this command from your customized shell (double-click on pyqgis.cmd to start it).

pb_tool

With pip installed we can use it to install pb_tool:

pip install pb_tool

More information on using pb_tool is available on the project website.

Working on the Command Line

Just double-click on your pyqgis.cmd script from the Explorer or a desktop shortcut to start a cmd shell. From here you can use Python interactively and also use pb_tool to compile and deploy your plugin for testing.

IDE Example

With slight modification, we can start our IDE with the proper settings to recognize the QGIS libraries:

@echo off
SET OSGEO4W_ROOT=C:\OSGeo4W
call "%OSGEO4W_ROOT%"\bin\o4w_env.bat
call "%OSGEO4W_ROOT%"\apps\grass\grass-6.4.3\etc\env.bat
@echo off
path %PATH%;%OSGEO4W_ROOT%\apps\qgis\bin
path %PATH%;%OSGEO4W_ROOT%\apps\grass\grass-6.4.3\lib

set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\qgis\python;
set PYTHONPATH=%PYTHONPATH%;%OSGEO4W_ROOT%\apps\Python27\Lib\site-packages
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
set PATH=C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\Vim\vim74;%PATH%
cd %HOMEPATH%\development
start "PyCharm aware of Quantum GIS" /B "C:\Program Files (x86)\JetBrains\PyCharm 3.4.1\bin\pycharm.exe" %*

We only changed the last line, adding the start statement with the path to the IDE (PyCharm). If you save this to something like pycharm.cmd, you can double-click on it to start PyCharm. The same method works for other IDEs, such as PyDev.

Within your IDE settings, point it to use the Python interpreter included with OSGeo4W---typically at: %OSGEO4W_ROOT%\bin\python.exe. This will make it pick up all the QGIS goodies needed for development, completion, and debugging. In my case OSGEO4W_ROOT is C:\OSGeo4W, so in the IDE, the path to the correct Python interpreter would be: C:\OSGeo4W\bin\python.exe.

Make sure you adjust the paths in your .cmd scripts to match your system and software locations.

Workflow

Here is an example of a workflow you can use once you're setup for development.

Creating a New Plugin

  1. Use the Plugin Builder plugin to create a starting point [1]
  2. Start your pyqgis.cmd shell
  3. Use pb_tool to compile and deploy the plugin (pb_tool deploy will do it all in one pass)
  4. Activate it in QGIS and test it out
  5. Add code, deploy, test, repeat

Working with Existing Plugin Code

The steps are basically the same was creating a new plugin, except we start by using pb_tool to create a new config file:

  1. Start your pyqgis.cmd shell
  2. Change to the directory containing your plugin code
  3. Use pb_tool create to create a config file
  4. Edit pb_tool.cfg to adjust/add things create may have missed
  5. Start at step 3 in Creating a New Plugin and press on

Troubleshooting

Assuming you have things properly installed, trouble usually stems from an incorrect environment.

  • Make sure QGIS runs and the Python console is available and working
  • Check all the paths in your pygis.cmd or your custom IDE cmd script
  • Make sure your IDE is using the Python interpreter that comes with OSGeo4W


[1] Plugin Builder 2.6 will support generation of a pb_tool config file

PyQGIS Resources

Here is a short list of resources available when writing Python code in QGIS. If you know of others, please leave a comment.

Blogs/Websites

In alphabetical order:

Documentation

Example Code

  • Existing plugins can be a great learning tool
  • Code Snippets in the PyQGIS Cookbook

Plugins/Tools

  • Script Runner: Run scripts to automate QGIS tasks
  • Plugin Builder: Create a starter plugin that you can customize to complete your own plugin
  • Plugin Reloader: Allows you to reload a plugin from within QGIS
  • pb_tool: Tool to compile and deploy your plugins

Books

QGIS Development with Plugin Builder and pb_tool

The Plugin Builder is a great tool for generating a working plugin project that you can customize.

One of the main tasks in the development cycle is deploying the plugin to the QGIS plugin directory for testing. Plugin Builder comes with a Makefile that can be used on Linux and OS X to aid in development. Depending on your configuration, the Makefile may work on Windows.

To help in managing development of your projects, we've come up with another option---a Python tool called pb_tool, which works anywhere QGIS runs.

Here's what it provides:

Usage: pb_tool [OPTIONS] COMMAND [ARGS]...

  Simple Python tool to compile and deploy a QGIS plugin. For help on a
  command use --help after the command: pb_tool deploy --help.

  pb_tool requires a configuration file (default: pb_tool.cfg) that declares
  the files and resources used in your plugin. Plugin Builder 2.6.0 creates
  a config file when you generate a new plugin template.

  See http://g-sherman.github.io/plugin_build_tool for for an example config
  file. You can also use the create command to generate a best-guess config
  file for an existing project, then tweak as needed.

Options:
  --help  Show this message and exit.

Commands:
  clean       Remove compiled resource and ui files
  clean_docs  Remove the built HTML help files from the...
  compile     Compile the resource and ui files
  create      Create a config file based on source files in...
  dclean      Remove the deployed plugin from the...
  deploy      Deploy the plugin to QGIS plugin directory...
  doc         Build HTML version of the help files using...
  list        List the contents of the configuration file
  translate   Build translations using lrelease.
  validate    Check the pb_tool.cfg file for mandatory...
  version     Return the version of pb_tool and exit
  zip         Package the plugin into a zip file suitable...

In the command summary, a description ending in ... means there is more to see using the help switch:

pb_tool zip --help
Usage: pb_tool zip [OPTIONS]

  Package the plugin into a zip file suitable for uploading to the QGIS
  plugin repository

Options:
  --config TEXT  Name of the config file to use if other than pb_tool.cfg
  --help         Show this message and exit.

The Configuration File

pb_tool relies on a configuration file to do its work. Here's a sample pb_tool.cfg file:

# Configuration file for plugin builder tool
# Sane defaults for your plugin generated by the Plugin Builder are
# already set below.
#
[plugin]
# Name of the plugin. This is the name of the directory that will
# be created in .qgis2/python/plugins
name: TestPlugin

[files]
# Python  files that should be deployed with the plugin
python_files: __init__.py test_plugin.py test_plugin_dialog.py

# The main dialog file that is loaded (not compiled)
main_dialog: test_plugin_dialog_base.ui

# Other ui files for dialogs you create (these will be compiled)
compiled_ui_files: foo.ui

# Resource file(s) that will be compiled
resource_files: resources.qrc

# Other files required for the plugin
extras: icon.png metadata.txt

# Other directories to be deployed with the plugin.
# These must be subdirectories under the plugin directory
extra_dirs:

# ISO code(s) for any locales (translations), separated by spaces.
# Corresponding .ts files must exist in the i18n directory
locales: af

[help]
# the built help directory that should be deployed with the plugin
dir: help/build/html
# the name of the directory to target in the deployed plugin
target: help

The configuration file is pretty much self-explanatory and represents that generated by Plugin Builder 2.6 for a new plugin. As you develop your code, you simply add the file names to the appropriate sections.

Plugin Builder 2.6 will be available the week of the QGIS 2.6 release. In the meantime, you can use pb_tool create to create a config file. See the pb_tool website for more information.

Deploying

Here's what a deployment looks like with pb_tool:

$ pb_tool deploy
Deploying will:
            * Remove your currently deployed version
            * Compile the ui and resource files
            * Build the help docs
            * Copy everything to your .qgis2/python/plugins directory

Proceed? [y/N]: y
Removing plugin from /Users/gsherman/.qgis2/python/plugins/TestPlugin
Deploying to /Users/gsherman/.qgis2/python/plugins/TestPlugin
Compiling to make sure install is clean
Skipping foo.ui (unchanged)
Compiled 0 UI files
Skipping resources.qrc (unchanged)
Compiled 0 resource files
Building the help documentation
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v1.2b1
loading pickled environment... done
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
no targets are out of date.

Build finished. The HTML pages are in build/html.
Copying __init__.py
Copying test_plugin.py
Copying test_plugin_dialog.py
Copying test_plugin_dialog_base.ui
Copying foo.py
Copying resources_rc.py
Copying icon.png
Copying metadata.txt
Copying help/build/html to /Users/gsherman/.qgis2/python/plugins/TestPlugin/help

Getting Started

For details on installing and using pb_tool, see: http://g-sherman.github.io/pluginbuildtool

The PyQGIS Programmer's Guide

The PyQGIS Programmer's Guide is now available in both paperback and PDF. A sample chapter is also available for download.

The book is fully compatible with the QGIS 2.x series of releases.

Plugin Builder 2.8

Plugin Builder 2.8 is now available. This is a minor update that adds:

  • Suggestion for setting up an issue tracker and creating a code repository
  • Suggestion for a home page
  • Tag selection from a list of current tags
  • Documentation update, including information about using pb_tool to compile, deploy, and package your plugin
  • New URLs for Plugin Builder's home page and bug tracking

Optional is now Recommended

In previous versions the following items were "Optional" when creating a new plugin:

  • Bug tracker
  • Home page
  • Repository
  • Tags

We've changed those from "Optional" to "Recommended" because they are important for the success and longevity of your plugin(s). Setting up a code repository on GitHub automatically gives you issue tracking and the ability for others to collaborate with fixes and enhancements through pull requests.

Using GitHub also gives you the ability to setup a home page right from your repository using GitHub pages.

Adding one or more tags to your plugin helps people find them easier when browsing the QGIS Plugins website.

Getting It

You can install Plugin Builder 2.8 from the Plugins -> Manage and Install Plugins... menu. Version 2.8 works on QGIS versions 2.0 and up.

Plugin Builder 2.8.1

This minor update to the Plugin Builder allows you to choose where your plugin menu will be located.

Previously your menu was placed under the Plugins menu. At version 2.8.1 you can choose from the following main menu locations:

  • Plugins
  • Database
  • Raster
  • Vector
  • Web

Plugins is the default choice when you open Plugin Builder. The value you choose is also written to the category field in your metadata.txt file. When you view your plugin in the Plugin Manager, the value of category is displayed, aiding folks in finding the menu location.

You can install Plugin Builder 2.8.1 from the Plugins -> Manage and Install Plugins... menu. Version 2.8.1 works on QGIS versions 2.0 and up.

Faking a Data Provider with Python

QGIS data providers are written in C++, however it is possible to simulate a data provider in Python using a memory layer and some code to interface with your data.

Why would you want to do this? Typically you should use the QGIS data providers, but here are some reasons why you may want to give it a go:

  • There is no QGIS data provider
  • The generic access available through OGR doesn't provide all the features you need
  • You have no desire to write a provider in C++
  • No one will write a C++ provider for you, for any amount of money

If you go this route you are essentially creating a bridge that connects QGIS and your data store, be it flat file, database, or some other binary format. If Python can "talk" to your data store, you can write a pseudo-provider.

To illustrate the concept, we'll create a provider for CSV files that allows you to create a layer and have full editing capabilities using QGIS and the Python csv module.

The provider will:

  • Create a memory layer from a CSV file
  • Create fields in the layer based on integer, float, or string values in the CSV
  • Write changes back to the CSV file
  • Require the CSV file to have an X and Y field
  • Support only Point geometries

We'll use the cities.shp file that comes with the PyQGIS Programmer's Guide to create a CSV using ogr2ogr:

ogr2ogr -lco "GEOMETRY=AS_XY" -f CSV cities.csv ~/Downloads/pyqgis_data/cities.shp

This gives us a CSV file with point geometry fields.

If you don't want to roll your own, you can download the cities.csv file here.

Here's the header (first row) and a couple of rows from the file:

X,Y,NAME,COUNTRY,POPULATION,CAPITAL
33.086040496826172,68.963546752929688,Murmansk,Russia,468000,N
40.646160125732422,64.520668029785156,Arkhangelsk,Russia,416000,N
30.453327178955078,59.951889038085938,Saint Petersburg,Russia,5825000,N

Signals and Slots

We'll look at the code in a moment, but here is the general flow and connection of signals from the layer to slots (methods) in our provider:

These are all the connections we need (signal -> slot) to implement editing of both geometries and attributes for the layer.

The Plugin Code

The plugin is pretty simple; when the tool is clicked, it opens a dialog allowing you to select a CSV file, then uses the CsvLayer class to read the file, create the memory layer, make connections (signals -> slots), and add it to the QGIS canvas.

The run Method

Here's the run method from the plugin (csv_provider.py):

 1 def run(self):
 2     """Run method that performs all the real work"""
 3     # show the dialog
 4     self.dlg.show()
 5     # Run the dialog event loop
 6     result = self.dlg.exec_()
 7     # See if OK was pressed
 8     if result:
 9         csv_path = self.dlg.lineEdit.text()
10         self.csvlayer = CsvLayer(csv_path)

The dialog (csv_provider_dialog.py) contains an intimidating warning and allows you to select a CSV file to load (lines 4-6).

Line 9 gets the path of the selected file from the dialog and line 10 creates the layer using the selected file. From that point on, you interact with the layer using the regular QGIS editing tools. We need to keep a reference to the layer (self.csvlayer), otherwise all our connections get garbage collected and we lose the "link" to the CSV file.


The CsvLayer Class

The CsvLayer class manages the creation, loading, and editing of the CSV file. First let's look at the methods in csv_layer.py that create the layer from our CSV file.

Creating the Layer from the CSV

The __init__ method examines the header and a sample row to determine the names and field types to be created in the layer. To read the CSV file, we use the csv module.

Here's the __init__ method:

 1 def __init__(self, csv_path):
 2     """ Initialize the layer by reading the CSV file, creating a memory
 3     layer, and adding records to it """
 4     # Save the path to the file so we can update it in response to edits
 5     self.csv_path = csv_path
 6     self.csv_file = open(csv_path, 'rb')
 7     self.reader = csv.reader(self.csv_file)
 8     self.header = self.reader.next()
 9     logger(str(self.header))
10     # Get sample
11     sample = self.reader.next()
12     self.field_sample = dict(zip(self.header, sample))
13     logger("sample %s" % str(self.field_sample))
14     field_name_types = {}
15     # create dict of fieldname:type
16     for key in self.field_sample.keys():
17         if self.field_sample[key].isdigit():
18             field_type = 'integer'
19         else:
20             try:
21                 float(self.field_sample[key])
22                 field_type = 'real'
23             except ValueError:
24                 field_type = 'string'
25         field_name_types[key] = field_type
26     logger(str(field_name_types))
27     # Build up the URI needed to create memory layer
28     self.uri = self.uri = "Point?crs=epsg:4326"
29     for fld in self.header:
30         self.uri += '&field={}:{}'.format(fld, field_name_types[fld])
31 
32     logger(self.uri)
33     # Create the layer
34     self.lyr = QgsVectorLayer(self.uri, 'cities.csv', 'memory')
35     self.add_records()
36     # done with the csv file
37     self.csv_file.close()
38 
39     # Make connections
40     self.lyr.editingStarted.connect(self.editing_started)
41     self.lyr.editingStopped.connect(self.editing_stopped)
42     self.lyr.committedAttributeValuesChanges.connect(self.attributes_changed)
43     self.lyr.committedFeaturesAdded.connect(self.features_added)
44     self.lyr.committedFeaturesRemoved.connect(self.features_removed)
45     self.lyr.geometryChanged.connect(self.geometry_changed)
46 
47     # Add the layer the map
48     QgsMapLayerRegistry.instance().addMapLayer(self.lyr)

Once the CSV file is opened, the header is read in line 8, and a sample of the first row is read in line 11.

Line 12 creates a dict that maps the field names from the header to the corresponding values in the sample row.

Line 16-25 look at each sample value and determine if its type: integer, real, or string.

Lines 28-30 create the URI needed to create the memory layer, which is done in line 34.

The add_records method is called to read the CSV and add the features in line 35.

Lastly we make the connections needed to support editing of the attribute table and the CSV file in response to the actions in our Signal/Slot diagram (lines 40-45).

Here is the add_records method that reads the CSV and creates a corresponding feature in the newly created memory layer:

 1 def add_records(self):
 2     """ Add records to the memory layer by reading the CSV file """
 3     # Return to beginning of csv file
 4     self.csv_file.seek(0)
 5     # Skip the header
 6     self.reader.next()
 7     self.lyr.startEditing()
 8 
 9     for row in self.reader:
10         flds = dict(zip(self.header, row))
11         # logger("This row: %s" % flds)
12         feature = QgsFeature()
13         geometry = QgsGeometry.fromPoint(
14             QgsPoint(float(flds['X']), float(flds['Y'])))
15 
16         feature.setGeometry(geometry)
17         feature.setAttributes(row)
18         self.lyr.addFeature(feature, True)
19     self.lyr.commitChanges()

Each row in the CSV file is read and both the attributes and geometry for the feature are created and added to the layer.

Managing Changes

With the layer loaded and the connections made, we can edit the CSV using the regular QGIS editing tools. Here is the code for the connections that make this happen:

 1 def editing_started(self):
 2     """ Connect to the edit buffer so we can capture geometry and attribute
 3     changes """
 4     self.lyr.editBuffer().committedAttributeValuesChanges.connect(
 5         self.attributes_changed)
 6 
 7 def editing_stopped(self):
 8     """ Update the CSV file if changes were committed """
 9     if self.dirty:
10         logger("Updating the CSV")
11         features = self.lyr.getFeatures()
12         tempfile = NamedTemporaryFile(mode='w', delete=False)
13         writer = csv.writer(tempfile, delimiter=',')
14         # write the header
15         writer.writerow(self.header)
16         for feature in features:
17             row = []
18             for fld in self.header:
19                 row.append(feature[feature.fieldNameIndex(fld)])
20             writer.writerow(row)
21 
22         tempfile.close()
23         shutil.move(tempfile.name, self.csv_path)
24 
25         self.dirty = False
26 
27 def attributes_changed(self, layer, changes):
28     """ Attribute values changed; set the dirty flag """
29     if not self.doing_attr_update:
30         logger("attributes changed")
31         self.dirty = True
32 
33 def features_added(self, layer, features):
34     """ Features added; update the X and Y attributes for each and set the
35     dirty flag
36     """
37     logger("features added")
38     for feature in features:
39         self.geometry_changed(feature.id(), feature.geometry())
40     self.dirty = True
41 
42 def features_removed(self, layer, feature_ids):
43     """ Features removed; set the dirty flag """
44     logger("features removed")
45     self.dirty = True
46 
47 def geometry_changed(self, fid, geom):
48     """ Geometry for a feature changed; update the X and Y attributes for
49     each """
50     feature = self.lyr.getFeatures(QgsFeatureRequest(fid)).next()
51     pt = geom.asPoint()
52     logger("Updating feature {} ({}) X and Y attributes to: {}".format(
53         fid, feature['NAME'], pt.toString()))
54     self.lyr.changeAttributeValue(fid, feature.fieldNameIndex('X'),
55                                   pt.x())
56     self.lyr.changeAttributeValue(fid, feature.fieldNameIndex('Y'),
57                                   pt.y())

The dirty flag is used to indicate that the CSV file needs to be updated. Since the format doesn't support random access to individual rows, we rewrite the entire file each time an update is needed. This happens in the editing_stopped method.

When attributes are changed and/or removed, the only action taken is to set the dirty flag to True.

When there is a geometry change for a feature, it is updated in the attribute table immediately to keep the X and Y values in sync with the feature's coordinates. This happens in the geometry_changed method.

You can view the full code for both the CsvLayer class and the plugin on GitHub or by installing the plugin. To install the plugin, you'll need to make sure you have the Show also experimental plugins checkbox ticked in the Plugin Manager settings. To help you find it, the plugin is listed as CSV Provider in the Plugin Manager.

Looking Further

A few things about this implementation:

  • It is an example, not a robust implementation
  • It lacks proper error handling
  • There is no help file
  • It could be extended to support other geometry types in CSV
  • In its current form, it may not work for other CSV files, depending on how they are formatted (take a look at the Add Delimited Text Layer dialog in QGIS to see what I mean)
  • If you want to enhance this for fun or profit---well for fun, fork the repository and give me some pull requests

A couple of final points:

  • Going this route can be a lot more work than using an existing data provider
  • This method can be (and has been) used successfully to interface with a data store for which there is no data provider

Game of Life – Raster edition

You probably remember my Game of Life posts from last year: Experiments with Conway’s Game of Life & More experiments with Game of Life where I developed a vector-based version of GoL.

Richard Wen and Claus Rinner at Ryerson University now published a raster-based version.

Here’s a screenshot of the script in action:

Screenshot 2015-03-08 20.04.07

The code is hosted on Github and I’m sure there will be many other ideas which can build on code snippets to read and write raster cell values.

For more info, please visit the GIS at Ryerson blog.


More experiments with Game of Life

As promised in my recent post “Experiments with Conway’s Game of Life”, I have been been looking into how to improve my first implementation. The new version which you can now find on Github is fully contained in one Python script which runs in the QGIS console. Additionally, the repository contains a CSV with the grid definition for a Gosper glider gun and the layer style QML.

Rather than creating a new Shapefile for each iteration like in the first implementation, this script uses memory layers to save the game status.

You can see it all in action in the following video:

(video available in HD)

Thanks a lot to Nathan Woodrow for the support in getting the animation running!

Sometimes there are still hick-ups causing steps to be skipped but overall it is running nicely now. Another approach would be to change the layer attributes rather than creating more and more layers but I like to be able to go through all the resulting layers after they have been computed.


Experiments with Conway’s Game of Life

This experiment is motivated by a discussion I had with Dr. Claus Rinner about introducing students to GIS concepts using Conway’s Game of Life. Conway’s Game of Life is a popular example to demonstrate cellular automata. Based on an input grid of “alive” and “dead” cells, new cell values are computed on each iteration based on four simple rules for the cell and its 8 neighbors:

  1. Any live cell with fewer than two live neighbours dies, as if caused by under-population.
  2. Any live cell with two or three live neighbours lives on to the next generation.
  3. Any live cell with more than three live neighbours dies, as if by overcrowding.
  4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

(Source: Wikipedia – Conway’s Game of Life)

Based on these simple rules, effects like the following “glider gun” can be achieved:

Gospers glider gun.gif
Gospers glider gun” by KieffOwn work. Licensed under CC BY-SA 3.0 via Wikimedia Commons.

There are some Game of Life implementations for GIS out there, e.g. scripts for ArcGIS or a module for SAGA. Both of these examples are raster-based. Since I couldn’t find any examples of raster manipulation like this in pyQGIS, I decided to instead implement a vector version: a Processing script which receives an input grid of cells and outputs the next iteration based on the rules of Game of Life. In the following screencast, you can see the Processing script being called repeatedly by a script from the Python console:

So far, it’s a quick and dirty first implementation. To make it more smooth, I’m considering adding spatial indexing and using memory layers instead of having Processing create a bunch of Shapefiles.

It would also be interesting to see a raster version done in PyQGIS. Please leave a comment if you have any ideas how this could be achieved.


Two book recommendations

I recently finished reading two books which may be of interest to open-source GIS users – “PostGIS Cookbook” and “The PyQGIS Programmer’s Guide“, both of which I highly recommend:

PostGIS Cookbook

PostGIS CookbookI’ve been a fan of Stephen Mather’s blog for a while now, and have consistently found it to be a great source of trustworthy information and creative solutions to GIS problems. So when I first saw mention of his work on the PostGIS Cookbook I knew it would be a must-read for me. PostGIS is an essential part of my daily toolkit, and I’ll quickly devour any tutorial or guide which can lead me to better ways to put it to use. And that’s exactly what this book is! It’s full of tips and guides which has inspired me in a lot of techniques I’d never tried or even thought possible in PostGIS.

It’s important to point out that this book isn’t a training manual or beginner’s guide to PostGIS. It assumes readers are already familiar with using PostGIS and have a good understanding of GIS software in general. (If you’re looking for a book to start from scratch with PostGIS, PostGIS in Action is a better fit). I think that’s really what makes this book stand out though. There’s currently not a lot of books available covering PostGIS, and as far as I’m aware the PostGIS Cookbook is the only book available which is targeted to experienced PostGIS users.

Highlights for me are:

  • A great explanation and write up on optimised KNN filtering in PostGIS (something which often trips me up)
  • The detailed guide to topologically correct simplification of features
  • The exploration of PgRouting, which is a great introduction to PostGIS’ routing abilities
  • The “PostGIS and the web” chapter – I really wasn’t expecting this, but it’s quite eye opening (I’m going to have to do some digging into GeoDjango sometime)

The only criticism I have with this book is that it jumps around a lot between operating systems. While most of the code is provided for both Linux/OSX and Windows, there’s occasional examples which only have code for one specific operating system. It’s a little jarring and assumes the user is well versed in their particular operating system to workaround these omissions.

Overall, I strongly recommend the PostGIS Cookbook, and would consider it a must have for anyone serious about expanding their PostGIS abilities. (Also, looks like the publisher, Packt, have a two-for-one sale going at the moment, so it’s a good time to grab this title).

The PyQGIS Programmer’s Guide

The PyQGIS Programmer's GuideThe second book I’ve just finished reading is Gary Sherman’s “The PyQGIS Programmer’s Guide“. For those who are unaware, Gary was the original founder of QGIS back in 2002, so you can be confident that he knows exactly what he’s writing about. In The PyQGIS Programmer’s Guide  Gary has created an in-depth guide on how to get started with programming for QGIS using python. It takes readers all the way from simple scripts right through to developing QGIS plugins and standalone applications based on the QGIS API.

This book fills an important void in the literature available for QGIS. Previously, the PyQGIS Developer Cookbook was the only available guide for QGIS python scripting, and unfortunately it’s a little out-of-date now. PyQGIS scripting can be a steep learning curve and that’s why this book is so appreciated.

It would be valuable to have some python knowledge and experience prior to reading this book. While the “Python Basics” chapter quickly runs through an introduction to the language, the book makes no claims to be a comprehensive python tutorial. But if you’ve dabbled in the language before and have familiarity with the python way of doing things you’ll easily be able to follow along.

Highlights are:

  • The “Tips and Techniques” chapter, which is a great mini-reference for performing a range of common tasks in PyQGIS (including loading layers, changing symbol styles, editing feature attributes, etc).
  • A complete tutorial for creating a QGIS plugin
  • A guide to debugging PyQGIS code and plugins

I’d definitely recommend that anyone who wants to get started with PyQGIS start with Gary’s work – you’ll find it the perfect place to begin.

A routing script for the Processing toolbox

Did you know that there is a network analysis library in QGIS core? It’s well hidden so far, but at least it’s documented in the PyQGIS Cookbook. The code samples from the cookbook can be used in the QGIS Python console and you can play around to get a grip of what the different steps are doing.

As a first exercise, I’ve decided to write a Processing script which will use the network analysis library to create a network-based route layer from a point layer input. You can find the result on Github.

You can get a Spatialite file with testdata from Github as well. It contains a network and a routepoints1 layer:

points_to_route1

The interface of the points_to_route tool is very simple. All it needs as an input is information about which layer should be used as a network and which layer contains the route points:

points_to_route0

The input points are considered to be ordered. The tool always routes between consecutive points.

The result is a line layer with one line feature for each point pair:

points_to_route2

The network analysis library is a really great new feature and I hope we will see a lot of tools built on top of it.


Generating chainage (distance) nodes in QGIS

Something that I need to do now and then is generate points along a line at supplied distance.  I had never really looked into doing it in QGIS until this question poped up on gis.stackexchange.com.  This is a quick blog post because I thought it was a pretty handy little thing to do.

In the development version there is a new method on QgsGeometry called interpolate. This method takes a distance and returns a point along a line at that distance. Perfect! We can then just wrap this in a loop and generate a point increasing the distance as we move along

from qgis.core import (QgsFeature, QgsGeometry,
                       QgsVectorLayer, QgsMapLayerRegistry,
                       QgsField)
from PyQt4.QtCore import QVariant
from qgis.utils import iface

def createPointsAt(distance, geom):
    length = geom.length()
    currentdistance = distance
    feats = []

    while currentdistance < length:
        # Get a point along the line at the current distance
        point = geom.interpolate(currentdistance)
        # Create a new QgsFeature and assign it the new geometry
        fet = QgsFeature()
        fet.setAttributeMap( { 0 : currentdistance } )
        fet.setGeometry(point)
        feats.append(fet)
        # Increase the distance
        currentdistance = currentdistance + distance

    return feats

def pointsAlongLine(distance):
    # Create a new memory layer and add a distance attribute
    vl = QgsVectorLayer("Point", "distance nodes", "memory")
    pr = vl.dataProvider()
    pr.addAttributes( [ QgsField("distance", QVariant.Int) ] )
    layer = iface.mapCanvas().currentLayer()
    # Loop though all the selected features
    for feature in layer.selectedFeatures():
        geom = feature.geometry()
        features = createPointsAt(distance, geom)
        pr.addFeatures(features)
        vl.updateExtents()

    QgsMapLayerRegistry.instance().addMapLayer(vl)

The above code might look a bit scary at first if you have never done any Python/pyqgis but hopefully the comments will ease the pain a little. The main bit is the createPointsAt function.

Cool! If we want to use this we can just stick it in a file in the .qgis/python folder (lets call it pointtools.py) and then run import pointtools in the python console.

So lets have a go. First select some objects then run the following in the Python Console

import pointtools
pointtools.pointsAlongLine(40)

That will generate a point every 40 meters along then selected lines

Distance nodes along line in qgis-dev

To generate nodes along different lines, select the new features, then call pointtools.pointsAlongLine(40) in the Python console.

Simple as that!

(Who knows, someone (maybe me) might even add this as a core object function in QGIS in the future)


Filed under: Open Source, qgis Tagged: language python, Open Source, osgeo, pyqgis, python, qgis, qgis-tips, Quantum GIS, tips

How to Install Sphinx

This post summarizes how to install sphinx on Windows to contribute to PyQGIS Cookbook. I’m writing this as I go, so this most likely won’t be perfect.

I used my Python 2.6 stand-alone installation (not the one in OSGeo4W).

  1. Get the Sphinx egg from http://pypi.python.org/pypi/Sphinx
  2. If you don’t have it, install setuptools to get the easy_install script http://pypi.python.org/pypi/setuptools
  3. In C:\Python26\Scripts run easy_install -U sphinx
  4. Get the PyQGIS source from https://github.com/qgis/QGIS-Developer-Cookbook
  5. Create a build folder inside the QGIS-Developer-Cookbook
  6. Now you can build the Cookbook: sphinx-build "C:\Users\Anita\QGIS\QGIS-Developer-Cookbook\source" "C:\Users\Anita\QGIS\QGIS-Developer-Cookbook\build"

The build folder should now contain the Cookbook .html files.


How to Add a Legend to Your PyQGIS Application

In his post “Layer list widget for PyQGIS applications”, German Carrillo describes how to add a legend to your own PyQGIS application. You get to download the source code for the legend widget to include into your project.


  • Page 1 of 1 ( 18 posts )
  • pyqgis

Back to Top

Sponsors