top logo
 
  • Page 1 of 6 ( 120 posts )
  • >>
  • gis

Tags

Last update:
Mon May 20 11:00:26 2013

A Django site.

Print Composer 2.0 – Take #7

Today’s post: More print composer overview magic!

Inverted Map Overviews

Thanks to the “Invert overview” option, we can now chose between highlighting the detail area (left example in the image) or blocking out the surrounding area (right example).

printcomposer_overviews

The “Lock layers for map item” option can come in very handy if you want to reduce the number of layers in the overview map while still keeping all layers of interest in the main map.



QGIS gets Oracle support!

Seems this is a good day for QGIS Oracle users. According this commit made by Jürgen QGIS now has built-in Oracle support. Win!

Native Oracle support can now see QGIS being opened up to a wider user base yet again. A large user base normally means more people willing to sponsor awesome new features and bug fixes. Having seen the growth in the user base from having native MS SQL Server 2008+ support I can imagine what it will be like with Oracle.

The list of formats QGIS can open and edit is getting larger and larger with each release. Is there a future for vendor lock in? I can’t see it.

Standard disclaimer about latest dev build and new features :)


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, oracle, osgeo, qgis, qgis-editing, Quantum GIS


New QGIS PDF and HTML manuals

A quick update from the QGIS documentation team today on the mailing list. The QGIS 1.8 manual is now available in HTML and PDF form.

HTML manual can be found at:
http://docs.qgis.org/html/en/docs/user_manual/index.html

PDF manual at:
http://docs.qgis.org/pdf/QGIS-1.8-UserGuide-en.pdf

This has been part of an ongoing effort from the documentation team since before the 1.8 release to bring our all our documentation into reStructedText rather then LaTeX. Moving to reStructedText allows quicker updates and a larger range of final output formats.

I would like to thank everyone who has been involved in this process as I know what a grueling process updating documentation can be.

Community notice

Just remember you don’t have to be a programmer to contribute to an open source project. If you think you could contribute to the updating of the QGIS documentation please contact the team on the mailing list.


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, osgeo, qgis, Quantum GIS


Using a QGIS spatial index to speed up your code

If you need to do any kind of spatial operations in QGIS using Python or C++ you really want them to be as fast a possible in order reduce the amount of time you make the user wait. Lets take the simple scenario of a recent question that was asked on gis.stackexchange; Summing up values of neighboring polygons?.

I went for the SQL approach as I like how quick SQL can express what you need to do, however SQL is not the only way to skin a cat as spatialthoughts has shown in his blog post. Here Ujaval has used Python to find the neighboring polygons of each feature. Running the script on a small dataset yields results in reasonable time however running it on a larger dataset can take a long time.

In order to check if a feature touches another you need to have two features to compare against each other. The simple way to do this is to create two loops where you check each feature against every other feature. Here is a quick code example of just that.

layer = qgis.utils.iface.activeLayer()
# Select all features along with their attributes
allAttrs = layer.pendingAllAttributesList()
layer.select(allAttrs)
# Get all the features to start
allfeatures = {feature.id(): feature for (feature) in layer}

def noindex():
        for feature in allfeatures.values():
                for f in allfeatures.values():
                        touches = f.geometry().touches(feature.geometry())
                        # It doesn't matter if we don't return anything it's just an example

import timeit
print "Without Index: %s seconds " % timeit.timeit(noindex,number=1)

So the above code is pretty simple, just loop each feature and check against every other feature. No worries. No worries at least until you run this on a large dataset then I think you can see the issue here. Running the above code on a layer with around 28000 features takes 1912.41 seconds – that’s 31 minutes. Holy crap!

Note: We put all the features of the layer into a dictionary as it will make lookup quicker in the later index example.

How can we speed up the above code? Lets take a gander at QgsSpatialIndex

QgsSpatialIndex to rule them all

QgsSpatialIndex is a wrapper around the open source SpatailIndex lib and uses a RTree for an index method. If you don’t know what an index is you can think of it like the index in a book – a pointer to a location in the book rather then having to scan every page to find a word.

There isn’t much to using QgsSpatialIndex just insert each QgsFeature and it handles the rest, when we need something out we just use the intersects method to return any features inside an area.

layer = qgis.utils.iface.activeLayer()
# Select all features along with their attributes
allAttrs = layer.pendingAllAttributesList()
layer.select(allAttrs)
# Get all the features to start
allfeatures = {feature.id(): feature for (feature) in layer}

def withindex():
        # Build the spatial index for faster lookup.
        index = QgsSpatialIndex()
        for f in allfeatures.values():
                index.insertFeature(f)

        # Loop each feature in the layer again and get only the features that are going to touch.
        for feature in allfeatures.values():
          # Get the ids of all the features in the index that are within
          # the bounding box of the current feature because these are the ones
          # that will be touching.
          ids = index.intersects(feature.geometry().boundingBox())
          for id in ids:
            f = allfeatures[id]
            if f == feature: continue
            touches = f.geometry().touches(feature.geometry())
            # It doesn't matter if we don't return anything it's just an example

import timeit
print "With Index: %s seconds " % timeit.timeit(withindex,number=1)

Running this code on our 28000 feature layer returns the results in 10 seconds. 31 minutes down to 10 seconds by just using a spatial index. Nice!

So the next time you need to do some spatial operations remember to use the handy QgsSpatialIndex in order to speed up your code. If you don’t want to use QgsSpatialIndex, or need some more flexiblity, you could even use the Python RTree module.

Full code

layer = qgis.utils.iface.activeLayer()

# Select all features along with their attributes
allAttrs = layer.pendingAllAttributesList()
layer.select(allAttrs)
# Get all the features to start
allfeatures = {feature.id(): feature for (feature) in layer}

def noindex():
	for feature in allfeatures.values():
		for f in allfeatures.values():
			touches = f.geometry().touches(feature.geometry())
			# It doesn't matter if we don't return anything it's just an example

def withindex():
	# Build the spatial index for faster lookup.
	index = QgsSpatialIndex()
	map(index.insertFeature, allfeatures.values())

	# Loop each feature in the layer again and get only the features that are going to touch.
	for feature in allfeatures.values():
	  ids = index.intersects(feature.geometry().boundingBox())
	  for id in ids:
	    f = allfeatures[id]
	    touches = f.geometry().touches(feature.geometry())
	    # It doesn't matter if we don't return anything it's just an example

import timeit
print "With Index: %s seconds " % timeit.timeit(withindex,number=1)
print "Without Index: %s seconds " % timeit.timeit(noindex,number=1)


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, qgis, Quantum GIS, spatial operations


Five new awesomely awesome QGIS features – Round 2

As QGIS is such a fast moving project I have decided to make this a regular blog post in order to highlight some new features added to QGIS. If you haven’t already, don’t forget to check out round one.

Remember that some of these features may still only be new which might change between now and the next official released version. With that out of the way lets get listing.

Atlas integration

If you are a regular user of QGIS Python plugins, and who isn’t, then you would have used the awesome Atlas plugin developed by Vincent Picavet. This great tool can be used to generate mapbooks, or an atlas as some people like to say, using a coverage layer and a print composer. What makes this even more awesome is that it is now built into the print composer.

Atlas composer intergration

The builtin atlas function also gives you the ability to use an expression to do runtime text replacement, including access to all the fields on the coverage layer.  The coverage layer doesn’t even have to be a region layer, it can be a simple point layer, or even a line layer.   You can see the result of me running the atlas generation from the above example here

Big thanks to Oslandia for integrating this great feature, and the companies sponsoring the work.

New Python console

This new addition comes from the great work that Salvatore Larosa has been doing to add a better Python console to QGIS.

The new Python console includes attribute auto complete, syntax highlighting, better copy and paste, uploading to codepad, the ability to run code from a file, etc.  You don’t realise how much difference there is until you go back to using the old one in version 1.8.

New Python console

Tabbed and groups in builtin forms

One of the things I really loved about QGIS, coming from MapInfo, was the builtin forms.  Just having the ability to enter data using controls like combo boxes, calendar widgets, etc makes you one step closer to having better data.   This feature is the exact reason I setup a 67 year old for kerb data collection.

As good as they are the builtin forms have an issue of ending up with as a big scrolling list with lots of fields; also the lack of  the ability to group or put fields on tabs in the UI meant you had to create a custom form.  Well not any more.

There is now a combo box on the Fields tab that allows you to build a generated form but also add tabs and group boxes.  You can even have the same field shown more then once on the form, handy for something like an ID field that you would like to show on each tab.

With this new ability the builtin forms can get me 95% of the way for data entry jobs, the other 5% I just make a custom form – but that is very rare.

Sextante

Sextante is a great and powerful analytical framework that has been added to the core of QGIS thanks to Victor Olaya.  This is not a feature that I use a lot but this is only due to most of my work being in development and not analysis, however that doesn’t mean that it’s not a really cool feature.

One of the greatest things about the Sextante toolbox is that it allows you to integrate other great open source tools like GRASS, SAGA, R, OTB, etc, right into your QGIS workflow and view the results in the canvas. It even includes a modeller so that you can build a connected diagram of all the bits of your process, even if it crosses between programs.

The toolbox

For me what is even better is that you can use Sextante in your plugins or custom Python code.  Sextante has a Python interface – well the whole thing is written in Python – that you can use to run a Sextante supported algorithm.

import sextante
outputs_0=sextante.runalg("grass:v.crossbones", /file, 0, ,, 1, 2, 3, 4, 1=3.0,2=8.0,5=6.0,8=6.0,11=6.0,14=6.0, None)
outputs_1=sextante.runalg("grass:v.delaunay", outputs_0['output'], True, True, None)
outputs_2=sextante.runalg("grass:v.dissolve", outputs_0['output'], , None)

Victor has created a blog to cover some Sextante recipes at QGIS-SEXTANTE cookbook/. There are also some really cool example of Sextante in use at:

Massive amount of composer additions

This last feature, or rather feature set,  comes from the sponsorship and support of the World BankAustralia-Indonesia Facility for Disaster Reduction, Geoscience Australia and the GFDRR.  Most of this work was done to aid in the development of a new QGIS plugin called inaSAFE, which has also received some great praise

“(InaSAFE) is very beneficial for all of us. It’s a good example of our partnership.”
Dr Susilo Bambang Yudhoyono – President of Indonesia

Some of the improvements include:

  • A new loadFromTemplate method, in the API, that can do text replacement in the QGIS composer template.
  • Better support for EPSG:4326 scale bars
  • Multipage in one composer
  • Automatic overview frame in map frame
  • HTML frame
  • Zebra style grid frame border
  • More control of grid labels. Inside, outside, disabled
  • etc, etc

These are great additions to the QGIS composer and I have already used the overview frame feature along with the new atlas integration  to make some quick nice looking map books. \

A huge thanks to the World BankAustralia-Indonesia Facility for Disaster ReductionGeoscience Australia and the GFDRR, and all the developers included.

You can see some of the output that InaSAFE generates using some of these new features at http://quake.linfiniti.com/

P.S The World Bank also sponsored the new raster Save As.. ability just like we have for vector layers.

 


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, osgeo, qgis, Quantum GIS


FOSS4G-AU in summary

Last Thursday and Friday was our first local Australian FOSS4G event which was hosted at the CSRIO building in Brisbane.  Very big thanks to CSRIO for hosting the event.  The venue was setup perfectly for  hosting an event like this, including dual projectors for presenting, video calls over to Perth, etc.

The first day was done using a un-conference style of event. This is the first time I’ve been to a un-conference and I liked the format a lot.   Once everyone was there on the first morning we collected ideas from people and everyone voted on which ones they would like to see.  After we had picked enough topics Shaun and I made a program for the day and we started.

Topics included:

Me presenting QMap

The second day was a code sprint.  I worked on converting a MapBasic scripts from one of the guys to QGIS, and Jody enlisted the others to help check the headers of the GeoServer project so that it can finally pass OSGeo incubation.

Overall I think it was a very successful event.  I would like to make these a yearly event if we can, provided that we have people to talk, or projects to work on.

More information about up coming OSGeo events in Australia and New Zealand can be found at http://www.meetup.com/osgeo-aust-nz/


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, open source gis, osgeo, qgis, Quantum GIS


Mapping OGDWien Population Density

The city of Vienna provides both subdistrict geometries and population statistics. Mapping the city’s population density should be straightforward, right? Let’s see …

We should be able to join on ZBEZ and SUB_DISTRICT_CODE, check! But what about the actual population counts? Unfortunately, there is no file which simply lists population per subdistrict. The file I found contains four lines for each subdistrict: females 2011, males 2011, females 2012 and males 2012. That’s not the perfect format for mapping general population density.

A quick way to prepare our input data is applying pivot tables, eg. in Open Office: The goal is to have one row per subdistrict and columns for population in 2011 and 2012:

Export as CSV, add CSVT and load into QGIS. Finally, we can join geometries and CSV table:

A quick look at the joined data confirms that each subdistrict now has a population value. But visualizing absolute values results in misleading maps. Big subdistricts with only average density will overrule smaller but much denser subdistricts:

That’s why we need to calculate population density. This is easy to do using Field Calculator. The subdistrict file already contains area values but even if they were missing, we could calculate it using the $area operator: "pop2012" / ($area / 10000). The resulting population density in population per ha finally shows which subdistricts are the most densely populated:

One could argue that this is still no accurate representation of population density: Big parts of some subdistricts are actually covered by water – especially along the Danube – and therefore uninhabited. There are also big parks which could be excluded from the subdistrict area. But that’s going to be the topic of another post.

If you want to use my results so far, you can download the GeoJSON file from Github.



User defined expression functions for QGIS

Ever since I added expression based labels, including the new expression builder UI, something that I always wanted to add is the ability to define custom user defined functions in Python (or C++) and use them in an expression. The expression engine is used for labels, rule based rendering, layer actions, field calculator, and atlas composer tags.  Thanks to the all the awesome work on the expression engine by Martin all this cool stuff is now possible.

Today I pushed a commit into master that adds the ability to define a function in Python (or C++), register it in the expression engine, then use it anywhere expressions are used.

The good stuff

Lets take a use case from Ujaval Gandhi and his example of counting vertices for each feature.

First we need to import the new qgsfunction decorator function from qgis.utils. The qgsfunction decorator will take a normal Python function, wrap it up in the class used to define a function, and register it in the engine.

So what does an empty function look like:

from qgis.utils import qgsfunction
from qgis.core import QGis

@qgsfunction(0, "Python")
def vertices(values, feature, parent):
	pass

@qgsfunction(0, "Python") means we are defining a new vertices function that takes 0 args and lives in the “python” group in the expression builder UI. Any custom function must take (values, feature, parent) as python args. values is a list of QVariants passed into the function, feature is the current QgsFeature, and parent is expression engine node (you use this to raise errors).

Lets stick some more logic in there:

from qgis.utils import qgsfunction
from qgis.core import QGis

@qgsfunction(0, "Python")
def vertices(values, feature, parent):
	"""
		Returns the number of vertices for a features geometry
	"""
	count = None
	geom = feature.geometry()
	if geom is None: return None
	if geom.type() == QGis.Polygon:
		count = 0
		if geom.isMultipart():
		  polygons = geom.asMultiPolygon()
		else:
		  polygons = [ geom.asPolygon() ]
		for polygon in polygons:
		  for ring in polygon:
		    count += len(ring)
	return count

Pretty simple. Get the geometry from the feature, check if it’s a polygon, if it is then count the number of vertices and return that number.

Now that we have that all done we can save it into a file in our .qgis/python folder, lets call it userfunctions.py (note you don’t have to save it here, anywhere that QGIS can find it will do.  Anywhere on PATH)

Lets open QGIS and run import userfunctions.py:

Importing functions from userfunctions.py

Now open the label properties for the layer:

The new function shown in the expression builder

Nice! Notice also that the function doc string is used as the function help. How cool is that.  You can also see the $ sign in front of the function, this is because any functions that take no args are considered special and use the $ sign as a convention, this is all automatic when the function is registered.

And the result is:

The label using the new function

You can even use it in the rule based rendering:

Rule rendering using new function

Enjoy!

Notes

  • You must unregister a function once you are finished with it using QgsExpression.unregisterFunction(name). This mainly applies to plugins where the user might unload your plugin and the code is no longer available. In the above example we could import userfunctions and never unregister because we plan on using it for the whole session.
  • You can’t override the built-in methods.

Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, OSS, qgis, Quantum GIS


WFS to PostGIS in 1 Step

This is an update to my previous post “WFS to PostGIS in 3 Steps”. Thanks to Even Rouault’s comments and improvements to GDAL, it is now possible to load Latin1-encoded WFS (like the one by data.wien.gv.at) into PostGIS in just one simple step.

To use the following instructions, you’ll have to get the latest GDAL (release-1600-gdal-mapserver.zip)

You only need to run SDKShell.bat to set up the environment and ogr2ogr is ready for action:

C:\Users\Anita>cd C:\release-1600-gdal-mapserver
C:\release-1600-gdal-mapserver>SDKShell.bat
C:\release-1600-gdal-mapserver>ogr2ogr -overwrite -f PostgreSQL PG:"user=myuser password=mypassword dbname=wien_ogd" "WFS:http://data.wien.gv.at/daten/geoserver/ows?service=WFS&request=GetFeature&version=1.1.0&typeName=ogdwien:BEZIRKSGRENZEOGD&srsName=EPSG:4326"

Thanks everyone for your comments and help!



Australian QGIS User Group

I’m very please to announce the Australian QGIS User Group. Chris Scott (DMS) and I created this group because the interest in QGIS here is growing fast and it would be nice to have a place where us Aussies can hang out and chat about QGIS stuff, discuss local issues, organize local events, put shrimps on the barbie.

My ultimate goal would be to turn this into a full user group much like the Swiss QGIS User Group but for now we will keep it light.

Everyone is free to view the content of the home page and the Google group, however you have to request to join the Google Group to post.  This is only so that we can keep it local and not reduce the official QGIS mailing lists.

So in saying all that. If you live in Australia. Use QGIS, or are interested in QGIS. Feel free to join the group.


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, osgeo, qgis, Quantum GIS


Don’t forget to migrate your QGIS plugins!

From QGIS 1.8 and onwards the Plugin Installer plugin will no longer include the option to add the 3rd party repositories.  This was by design and intended to move people over to using the official plugin repository at http://plugins.qgis.org/ so we can provide a richer experience and keep everything in one place.

If you have plugins that are still not on the official repository then I would strongly recommend that you migrate them as a lot of new 1.8 users will be missing out on your great work.


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, python, qgis, Quantum GIS


HTML map tips in QGIS

New fresh QGIS feature! So fresh in fact you can still smell the wet paint :)

QGIS (development build) can now display map tips using HTML (a subset anyway).

To enable the new map tips: Open the Layer Properties dialog for a layer and select the Display tab

Display tab to set HTML map tips

In action

Layer properties for HTML map tip

Notice how we can also use a QGIS expression. Anything inside [% %] will be evaluated and replaced with the value in real-time. We can even use a CASE statement. Pretty cool!

And the result when hovering over a feature

HTML in QGIS map tip? Yes! WOOT!

Hold on. Pause the track! We can even use some CSS to make it more fancy.


<style>
h1 {color:red;}
p.question {color:blue;}
</style>
<h1> [% "NAME" %] </h1>
<br>
<img src="[% "image" %]" />
<br>
<p class="question">Is this place a country?</p>
<br>
[% CASE WHEN "TYPE" = 'Country' THEN 'Yes' ELSE 'No. It is a ' || "TYPE" END %]

CSS in a html map tip

Happy Mapping :)


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, mapping, Open Source, qgis, styling, tips


Better date and time support in QGIS expressions and styles

Version note: This will only work in the latest dev build of QGIS – not in 1.8

The lack of uni for the next couple of weeks has left me some time at night to work on some features that I really wish QGIS had.  One of these features was better date and time support in the expression engine.  Date and time is an important concept when working on inspection data and not being able to style my features in QGIS using date operations was bugging me.  So in good open source fashion I added some.

Here are the current functions (more to come in the future):

  • $nowreturns the current date and time
  • age({datetime},{datetime}) - returns the difference between the two dates
  • todate({string}) - converts a string to date type
  • totime({string}) – converts a string to time type
  • tointerval({string}) – converts a string to a interval type (details below)
  • day({datetime} or {interval}) – returns the day from a datetime type or the number of days in a interval.
  • hour(…) – Same as above but for hours
  • minute(…)  - Same as above but for minutes
  • second(…)  - Same as above but for seconds
  • day(..)  - Same as above but for days
  • week(..)  - Same as above but for weeks
  • month(…)  - Same as above but for months
  • year(…) - Same as above but for years
  • {datetime} – {interval} = {new datetime} – returns a new datetime subtracting the interval 
  • {datetime} + {interval} = {new datetime} – returns a new datetime adding the interval


The interval type

Functions like age(..), tointerval(), {datetime} -/+ {interval}, day(..), hour(..), etc, use, or return, Intervals.  An Interval is a measure of time that we can use for different things.  An example of an Interval is ’1 Year 2 Months’ this is then converted to a number of seconds and used for any calculations.

For example one can take away 10 days from the current date by doing the following ( -> marks the output ):

todate($now - '10 Days')
-> 2012-06-20

as

todate($now)
-> 2012-06-30

We can also do something like:

todate($now + '2 Years 1 Month 10 Days')
-> 2014-08-10

The age() function will return an interval which we can use extract what information we need.

The number of days between two dates:

day(age('2012-06-30', '2012-06-10'))
-> 20
-- Think of it as '2012-06-30' - '2012-06-10'
-- Note: day(), month(), etc, functions return doubles so you can get
-- 21.135234 days if you are using date & time type rather than just date type
-- wrap the result in toint() to get a more sane output.

Day() will also work on a plain date:

day('2012-06-30')
-> 30

We can even get the number of seconds between two dates:

second(age('2012-06-30', '2012-06-10'))
-> 1728000

Currently the only date format supported is {year}-{month}-{day} as seen in the examples above. Shouldn’t be too hard to add support to the todate(), todatetime(), totime() functions for giving it a pattern to use when converting the string e.g. dd-mm-YYYY, or something like that.

More on this fancy new stuff

When I wrote the new expression builder dialog a while ago I made it dynamic so that any new functions added to the expression engine will show up automatically.  So here they are:

List of new date and time functions.

We can also use these functions in the rule based rending, which is where the power really comes in.  Lets see something like that in action:

Styled using days and years

Should be pretty straight forward to understand. We are using the age() and day() functions to style the events that are older than 30 days, within 30 days, for today, or in the future.  We also check the year of the event using year() and year($now) to make sure we only see this years events, or style them differently depending on if they are last years events or in the future.

This is the result of the above rules:

Result of using date functions in rule based renderer

I’m also using the date functions in the expression based labelling to label the features using the following expression:

CASE
WHEN year( "dateadded") < year($now) THEN
	'Last Year'
WHEN day(age("dateadded", $now)) < 0 THEN
	day(age("dateadded", todate($now))) || ' Days old'
ELSE
	day(age("dateadded", todate($now))) || ' Days to go'
END

Well that’s it. Hope you find it handy in your day-to-day mapping. I know I will be using it a lot.
Thanks to Martin and Jürgen for the code reviews during the process; venturing in an unknown part of the code base always makes me nervous but that is all part of learning, and sometimes you can make some pretty cool stuff.
Some other random notes: The general idea has been modelled of how Postgres handles dates and times, it’s not an exact copy but follows the same kind of ideas. The interval class also uses the same number of seconds for one year that postgres does so that we can be consistent with the output.


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, map-rendering, Open Source, osgeo, qgis, Quantum GIS, styling


Styling temporal (time) data in QGIS

So this years first uni semester is done and dusted, now I have some free time.  Blog all the things!

This is a follow up post for discussion that was started on LinkedIn about showing features older, or newer, then a certain date different colours   The main post was about using free, or low cost, solutions in order to aid in mapping water networks. I recommend that everyone watch it. A very good presentation.

I recommended that you could use the rule based rendering engine but the expression engine in QGIS doesn’t have any date functions yet.  All good we can add them if we need and once I get my head around the expression engine I plan on doing exactly that. But for now we can do it a different way.

We are going to use Spatialite, but any database will do (syntax and process will vary).

Lets have a look an inspection layer we have in our Spatialite database viewed in QGIS:

Pretty boring and hard to see what has been done in the last 30 days.  Now with the lack of support for dates in the expression engine we have to use another methods.  For this example we will use the really handy DBManger plugin that now ships with QGIS from 1.8.  Load it, connect to your database, and run the following query:

SELECT id,
              CASE WHEN DATE("date_checked") > DATE('now', '-30 days') THEN
                         'Within 30 Days'
              ELSE
                         'older'
              END as age, date_checked, geom
FROM  "inspections"

As you can see anything that is within the 30 days now has the “Within 30 Days” string in the age column, or else it has “older”.  CASE statements can be very powerful things in SQL sometimes.

Now load it into QGIS, style, and label it using the new age column

and Bob’s your uncle

You now have a layer that is style based on age but is also dynamic.  Adding a new inspection point will now will be styled according to those rules. (Although you will have to edit the normal inspection layer with it turned off as views/queries are not editable – without some setup anyway)

It might be a simple thing to some but sometimes it’s hard to find the right words to describe what you want when you are looking for this kind of thing. So hopefully this has helped a few people get started with visualizing their time/date based data in QGIS.

Happy mapping!


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, osgeo, qgis, Quantum GIS, styling


QGIS on Android using external GPS receivers

Thanks to FORNAT AG which sponsored me I could spend some time looking for solutions to make QGIS on android working with an external GPS receiver that sends NMEA strings.
It all boils down to the following:

  • via bluetooth works
  • via USB not yet (as soon as I get more sponsoring or time I’ll look more into it since I’ve some ideas).

To use an external bluetooth GPS you have to follow six easy steps:

  1. Turn of internal gps
  2. enable allow mock locations in settings>developer options
  3. install Bluetooth GPS app (there are others as well but I had some problems, this was the best for me)
  4. Pair the tablet and the gps receiver
  5. start Bluetooth GPS select enable mock locations provider and hit connect (here you can see if the connection works by checking the coords you see or even by looking on a google map)
  6. start QGIS, enable the GPS panel (view> panels > gps) and hit connect. that’s it

The only caveat is that you can’t check satellites and signal strength directly from within QGIS since the GPS data is relayed over the mock locations provider.

but for the rest it works as expected, now it would be cool if somehow it would be possible to connect a magic usb-to-bluetooth converter to our beloved usb only gps units…

ciao

 


Using QGIS in local government

Something that I always find interesting is how people are using different open source tools to get their work done.  This post attempts to outline how I/we are using QGIS at work for different projects.

Kerb mapping, condition, and defect pickup

This project is currently being done by a 67 year old foreman who has worked for the council for a very long time and has great knowledge of the town.   QGIS, with the main working layers stored in PostGIS, was setup so that he can:

  • Digitize kerb lines from aerial photos.
  • Split the existing kerb lines into segments depending on different asset rules.
  • Give each segment an overall condition rating.
  • Add defect points along the each kerb segment e.g. broken, lifted, etc,

Each defect point is snapped to the underlying  kerb line and chainages (distance along line) is generated using a update statement at the end of the project (could be done using a insert trigger if needed) using ST_line_locate_point(line, point).

Defect points coloured by risk captured against the kerb line

Overall QGIS has been great for this project.  The built in data entry forms have been a great help to allow fast and correct data entry. Each form has four drop downs all with present values and descriptions to aid in data entry.

Flood damaged claim maps

We recently suffered, like the rest of Queensland, some really major flooding which caused large amounts of damage to our road infrastructure. We got off pretty light compared to some places, nevertheless we still had a lot of damaged assets.  And so began the process of collecting data that could be used for state government funding claims.

Anyway, onto the QGIS bit.  QGIS was installed on one of the main engineers computers in order for him to make maps for each claim.  Having the ability for him to have one map window but multiple frames in the composer helped him to create multiple  views of the same data with ease.

In total there are 42 QGIS project files with a main project file which served the base layers to the other projects, using the cool Embed Layers and Groups feature.  This means any change in main base project was reflected up(down?) to the other projects next time they are opened.  The main project file has things like, property layer; normal road layers, with labels; road layer with roads for claims.   The other 42 projects have a filtered, and styled, road layer to only show roads in that batch, and its composers (print layouts).

Normally we would use MapInfo for this kind of thing but consider this: There are at least 3 print layouts per claim, each layout could have more then one map frame.  Now with MapInfo only being able to have a 1:1 ratio between the map window and the map frame in the layout you would need at least 3 map windows per claim.  Quick calc:

42 * 3 =  126+n map windows + 126 print layouts (n = extra map frames in layouts)

Each map window has its own copy of every layer, making change once apply every where changes hard.  This of course doesn’t apply to styles as they are stored in the .map (tab) file, but does for labels, style overrides, etc.   I’ll pass.

QGIS is no means perfect for printing or print layouts but the 1:N map window to map frame ratio worked really well for this project.  The styling options in QGIS also helped to change the display of the map depending on what was needed to be shown quickly, one even used the rule-based rendering.

You get the point.
Moving on.

Processing GPS photos with road chainages

This one I am quite proud of.  It’s nothing fancy but still saves a lot of time.  While not really QGIS only but a combination of QGIS+Spatialite it process GPS photos and assigns them a road name and chainage.

The issue: A large influx of GPS photos for the different flood damage projects and the need to process them quickly so that they got assigned to the correct road and chainage.  Now you can map GPS photos easy enough but then you still have to go to each one and assign a road name, chainage, and move it into the correct folder.  To hell with doing that by hand, this is why we invented GIS.

The result is a little (140 line) python script that:

  1. extracts the coordinates from each photos,
  2. finds the closest (within tolerance) road distance node (distance nodes are generated at 5m intervals along the road, around 800,000 in total for the whole shire),
  3. gets the road name, locality, and chainage for that node,
  4. creates a folder with that road name,
  5. renames the photo with {name} @ {chainage},
  6. moves it into the road name folder it is assign to.
  7. inserts a record for that photo into the spatialite database that can be viewed in QGIS.
The Spatialite database has a spatial index on the road distance nodes and with that in place it can process 148 photos in 8 seconds.  Not too bad at all. Now all we have to do to process the photos is stick them into a special folder and run process.bat.

Porting our planning scheme maps

I have been involved in creating, and maintaining, our planning scheme maps for the council.  It’s been a pretty fun project, apart from the constant moving target that is the state planning specifications, but I digress.

Planning scheme in QGIS

This project was done, and still is, in MapInfo. While there is nothing technically wrong with that, it has become a bit more of a pain to maintain then one would hope.  The planning scheme is not just one map but rather a series of different maps all with different scales and requirements.  I’m sure by now you can start to see the issues that can arise:

  1. No dynamic scale bar for layouts (not even a scale bar object rather just text and boxes made to look like a scale bar. With no group items feature moving these around is a pain).
  2. 1:1 map window to map frame means excessive map windows when the data is all the same with just different views.
  3. Legends don’t support ordering, adding items, or groups.
  4. With no embedding base maps feature like in QGIS it’s hard to change one thing and apply it to all the map windows/workspaces.
The specifications also ask for lines with symbols along them to show things like bikeways, footpaths etc, something that can’t be done in MapInfo, well it can by using the line style editor but I would rather stab myself in the eye.
The one thing I haven’t fully worked out how to do in QGIS yet is fully automate the printing process. Currently I open MapInfo using a batch file and pass it a workspace and MBX which prints the layouts and exits. I do this for each map type.    In QGIS I have a few options:
  1. Create a plugin that runs though each project and prints off its composers.
  2. Create a python script that runs from a batch file using qgis.core and qgis.gui QGIS python bindings.
  3. add a –code option to the command line of QGIS so that you could run: qgis.exe –noplugins –code “print.py”, which would open QGIS and run the python code and exit.
I’m yet to explore what option is the best for this project but I’ll get back to you on that.  Once I have the above issue sorted I plan on creating the maps in QGIS to see how it would turn out (time permitting)

Custom asset data collection program

This one would have to be my favourite.  I really love programming (most days), and being able to create our own data collection program using QGIS and MS SQL 2008 has been great.

While it is only very very young I’m already seeing some great potential.  Using an open source base (apart from MS SQL) has given us a lot of power, power to change stuff that we don’t like (which so far has been one minor bug), and the power to get exactly what we need.

I can’t talk about this project a lot as it is only very new and still only in design/testing/prototyping stage.

The main things for me are:

  • Ease of use. If I get asked how to do something over and over I have failed the users. And no 100 page training manuals.
  • Fast
  • No menus, or right-click menus! I’m a power user and even I hate navigating menus on a tablet.
  • Easy to build custom forms
  • Online/Offline syncing
  • Ease of use. Oh did I say this already!? Well it’s important.
  • Easy to configure by admins.
  • Limited use of dialogs. It’s NOT ok for an app to ask users to confirm 100 dialogs to do one thing.

Overall I think using QGIS and PyQt I can hit all the targets listed above quite well. In fact I know I can because I have already hit most of the them in the last couple of weeks.

Summary

So that is my list of QGIS uses in my local government situation, hopefully it wasn’t TL;DR and you found it interesting.  I’m sure there will be plenty more to add at the end of 2012.


Filed under: Open Source, qgis Tagged: case study, FOSSGIS, gis, local gov, Local government, mapinfo, Open Source, osgeo, qgis, Quantum GIS


QGIS Server on Windows7 Step-by-step

After my successful experiment with QGIS Server on Ubuntu, I took a shot at Windows7. These are my notes on installing QGIS Server following the instructions on the wiki.

Installation

Using OSGeo4W installer it is easy to install QGIS Server: Just mark qgis-server for installation from “Web” category (in the “Advanced” installation).

All other necessary packages will be selected automatically.

As mentioned on the wiki, the next step is to tell Apache which port number to use. Apache (2.2.14-4 from OSGeo4W) does not have any default IP/port set and it fails to start. To fix this, we need to edit the file

c:/osgeo4w/apache/conf/httpd.conf

and change

Listen @apache_port_number@

to our needs, e.g. to listen on port 80

Listen 80

The last thing I had to do to get QGIS Server working was to copy two files
libeay32.dll and ssleay32.dll
from C:\OSGeo4W\apache\bin
to C:\OSGeo4W\apps\qgis\bin.

The GetCapabilities request should work now

http://localhost/qgis/qgis_mapserv.fcgi.exe?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetCapabilities

Adding a QGIS project file

To add a project file to the server, we stay in C:\OSGeo4W\apps\qgis\bin. If we put a project file in this directory, it will be served by default (without having to pass the optional map parameter).

For this test, I added my vienna.qgs project file. This is how my QGIS bin folder looks like (notice the .dll files we copied from Apache/bin and the project file):

Next, we have to restart Apache to force QGIS Server to load the project file. The OSGeo4W installation provides a handy “Apache-Monitor” GUI to restart Apache. If it fails, try to reboot ;)

Let’s test the setup using “Add WMS Layer” in QGIS by adding the service URL and ticking “Ignore GetMap URI …” and “Ignore GetFeature URI …”.

The project layers are now available through WMS and can be loaded into your client.

QGIS "Add WMS Layer" dialog with my newly created WMS

Conclusion

That wasn’t bad. The wiki page was very helpful and I didn’t encounter any real problems. Editing a config file and copying a few .dlls is easy enough.

Since linking files is not one of Windows’ strengths, I’d expect a server with multiple projects to get quite messy. But it certainly works for home use and experiments.



QGIS on Android Phone

At FOSSGIS I was asked to try to install qgis on a very small android phone, I think it was a 3.2″ screen. the install went smoothly after making some space but then the problems came because of the small screen.
Eventually I thought about setting a smaller font size to make the UI scale more, the problem was that it was impossible to get to the size setting because the UI was to big.
As a workaround I created a QGIS.conf file with this content
[General]
IconSize=32
fontPointSize=4
and pushed it to the device using the android debug bridge like this:
adb push myQGISConfigFile.conf /data/data/org.qgis.qgis/files/Settings/QuantumGIS/QGIS.conf
On the next start the whole gui was nice and small and fitted the screen.

Here some screenshots from my Samsung galaxy 9000 with 4″ screen and a video demonstrating digitising (with pen and fingers), GPS, compass and zooming on the phone.

This video shows QGIS on a Samsung I9000 Galaxy S Android smartphone with 4.0″ screen. the point size in settings->option->general is set to 4

QGIS Multiview and globe screenshots

This screenshots have been created using the QGIS with the following plugins:

  • Multitemporal and multivariate data visualisation (http://hub.qgis.org/projects/multiview)
  • Scttergram identify (http://hub.qgis.org/projects/scattergramdentify
  • Globe Plugin
Screenshot-13 Screenshot-14 Screenshot-15 Screenshot-16 Screenshot-17 Screenshot-18 Screenshot-19 Screenshot-20 Screenshot-22 Screenshot-23 Screenshot-24 Screenshot-25 Screenshot-26 Screenshot-27 Screenshot-28 Screenshot-29 Screenshot-30 Screenshot-31 Screenshot-32 Screenshot-33 Screenshot-34 Screenshot-35 Screenshot-36 Screenshot-37 Screenshot-38


Custom QGIS feature forms – Value Binding

One thing I didn’t explain very well  in my other post was how to correctly set up value binding between your custom form and QGIS.  I didn’t explain it because at the time I didn’t know how.

The other day I was building a custom form QGIS for a project I am working on. I had named all the fields right, set the ui as the edit form for the layer, but only the line edits were getting bound to the correct values.

 So having a dig around in the code I noticed that QGIS uses the same methods to bind the built-in edit forms as it does for the custom forms, meaning that you must set what kind of control you want to use in Layer Properties -> Fields 

Correctly binding values

First create the form with the controls you need, remember to name them the same as your fields.

Custom form with controls using the same name as the fields

Note that here I have a QComboBox with the FeatureCla name, this will bind the combo box to the FeatureCla field in my dataset in QGIS.

Now set the custom form as the Edit UI for the layer

Set the Edit UI to your form

Tip: You can use relative paths if you store the form along side your project file

 Flick to the Fields tab and set up the Edit Widget type for each field that you have used on the custom feature form.

Set the Edit Widget that matches your control

I have set the FeatueCla field to use Unique values widget, this tells QGIS to collect all the unique values from that column and add them to the QComboBox.  There are a range of different edit widgets you can set

Each will map to a different set of control types (Widgets) e.g. If you want to have a checkbox on your form you must select Checkbox in the Edit Widget list to get it to bind correctly.

Save the properties and head back to you map.  Use the Identify Tool to select a feature.

Values bound to form

And that is it. Pretty cool hey!

Final thoughts

This is one feature I really like in QGIS.  The ability to create custom forms for people to do data entry without the need to build a plugin is very cool.  Couple this the built-in GPS module for QGIS and you have yourself a nice simple field data collection program.

I have some ideas to make this feature even more powerful, but more on that later once I get some time to add it in.


Filed under: Open Source, qgis Tagged: FOSSGIS, gis, Open Source, osgeo, qgis-editing, Quantum GIS

  • Page 1 of 6 ( 120 posts )
  • >>
  • gis

bottom

Powered by Django!