A lot of cartographers have a love/hate relationship with label halos. On one hand they can be an essential technique for improving label readability, especially against complex background layers. On the other hand they tend to dominate maps and draw unwanted attention to the map labels.
In this post I’m going to share my preferred techniques for using label halos. I personally find this technique is a good approach which minimises the negative effects of halos, while still providing a good boost to label readability. (I’m also going to share some related QGIS 3.0 news at the end of this post!)
Let’s start with some simple white labels over an aerial image:
These labels aren’t very effective. The complex background makes them hard to read, especially the “Winton Shire” label at the bottom of the image. A quick and nasty way to improve readability is to add a black halo around the labels:
Sure, it’s easy to read the labels now, but they stand out way too much and it’s difficult to see anything here except the labels!
We can improve this somewhat through a better choice of halo colour:
This is much better. We’ve got readable labels which aren’t too domineering. Unfortunately the halo effect is still very prominent, especially where the background image varies a lot. In this case it works well for the labels toward the middle of the map, but not so well for the labels at the top and bottom.
A good way to improve this is to take advantage of blending (or “composition”) modes (which QGIS has native support for). The white labels will be most readable when there’s a good contrast with the background map, i.e. when the background map is dark. That’s why we choose a halo colour which is darker than the text colour (or vice versa if you’ve got dark coloured labels). Unfortunately, by choosing the mid-toned brown colour to make the halos blend in more, we are actually lightening up parts of this background layer and both reducing the contrast with the label and also making the halo more visible. By using the “darken” blend mode, the brown halo will only be drawn for pixels were the brown is darker then the existing background. It will darken light areas of the image, but avoid lightening pixels which are already dark and providing good contrast. Here’s what this looks like:
The most noticeable differences are the labels shown above darker areas – the “Winton Shire” label at the bottom and the “Etheridge Shire” at the top. For both these labels the halo is almost imperceptible whilst still subtly doing it’s part to make the label readable. (If you had dark label text with a lighter halo color, you can use the “lighten” blend mode for the same result).
The only issue with this map is that the halo is still very obvious around “Shire” in “Richmond Shire” and “McKinlay” on the left of the map. This can be reduced by applying a light blur to the halo:
There’s almost no loss of readability by applying this blur, but it’s made those last prominent halos disappear into the map. At first glance you probably wouldn’t even notice that there’s any halos being used here. But if we compare back against the original map (which used no halos) we can see the huge difference in readability:
Compare especially the Winton Shire label at the bottom, and the Richmond Shire label in the middle. These are much clearer on our tweaked map versus the above image.
Now for the good news… when QGIS 3.0 is released you’ll no longer have to rely on an external illustration/editing application to get this effect with your maps. In fact, QGIS 3.0 is bringing native support for applying many types of live layer effects to label buffers and background shapes, including blur. This means it will be possible to reproduce this technique directly inside your GIS, no external editing or tweaking required!
At North Road we believe that crowdfunding is a sustainable way to maintain and enhance open source software, like the QGIS open source GIS package. We’ve run a number of successful crowdfunding campaigns in the past, including support in QGIS for live layer effects, a point cluster renderer, and a unique value renderer for raster layers.
Now, we’re proud to announce our latest crowd funding endeavour, and our biggest to date, the QGIS Layout and Reporting Engine Campaign.
This campaign covers stage 1 of a large, ongoing project to modernise and expand on QGIS’ print composer and layout facilities. Over time QGIS’ composer functionality has grown extensively and now is capable of creating flexible, high quality cartographic outputs. However, we’ve now hit a limit where the current code architecture is prohibiting further improvements and important fixes. In order to add a reporting framework to QGIS, it is necessary for us to refactor and improve large sections of the composer code.
If this campaign is successful, we’ll be adding flexible report generation features to QGIS and cleaning up all the existing composer code. As part of these clean up, we’ll be taking the opportunity to tackle a number of current limitations which cannot be addressed in the current composition code:
- Layouts will become unit aware, allowing for item placement and properties using millimetres, inches, pixels, centimetres, points, etc.
- Layouts will have the ability to include mixed page sizes and orientations.
- Plugins will be able to create custom composer item types (eg allow utilisation of 3rd party graphing and visualisation libraries!).
- Individual layout items can be rasterised without affecting the rest of the layout. For instance, a map which requires rasterisation due to its use of blend modes will not require all other layout items (such as headings, legends, etc) to be rasterised. This will greatly benefit PDF outputs for complex map layouts.
- The code refresh will allow more extensive use of data defined layout item properties.
- A render caching system will be implemented for items, speeding up use of the layout designer and also paving the way for use of live paint effects on layout items (eg dynamic drop shadows).
Full details on what we have planned are available here: QGIS Layout and Reporting Engine Campaign.
To make it possible we need 30,000€ pledged before 31 May 2017. You can help make this a reality by supporting the campaign or by sharing the campaign page and increasing exposure to the campaign. Updates to follow!
In 2012 I published a post on mapping the then newly released Tirol river dataset.
In the comments, reader Michal Zimmermann asked:
Do you think it would be possible to create a river stream which gains width along its way? I mean rivers are usually much narrower on their beginnings, then their width increases and the estuary should be the widest part, right?
With the help of geometry generators, we can now achieve this look directly in QGIS:
In the river dataset published by the state of Tirol, all rivers are digitized in upstream direction. For this styling to work, it is necessary that the line direction is consistent throughout the whole dataset.
We use a geometry generator symbol layer to split the river geometry into its individual segments:
Then we can use the information about the total number of segments (accessible via the expression variable @geometry_part_count) and the individual segment’s number (@geometry_part_num) to calculate the segment’s line width.
The stroke width expression furthermore uses the river category (GEW_GRKL) to vary the line width depending on the category:
CASE WHEN "GEW_GRKL" = '< 10 km2 Fluss' THEN 0.2 WHEN "GEW_GRKL" = '10 km2 Fluss' THEN 0.4 WHEN "GEW_GRKL" = '100 km2 Fluss' THEN 0.6 WHEN "GEW_GRKL" = '1.000 km2 Fluss' THEN 0.8 ELSE 1.0 END * ( 1- ( @geometry_part_num / @geometry_part_count ))
If the rivers are digitized in downstream direction, you can simply remove the 1- term.
Geometry generator symbol layers are a feature that has been added in QGIS 2.14. They allow using the expression engine to modify geometries or even create new geometries while rendering.
Geometry generator symbol layers make it possible to use expression syntax to generate a geometry on the fly during the rendering process. The resulting geometry does not have to match with the original geometry type and we can add several differently modified symbol layers on top of each other.
The latest version of the QGIS user manual provides some example expressions, which served as a basis for the following examples:
Rendering the centroid of a feature
To add a geometry layer representing feature centroids, we need to set the geometry type to Point / Multipoint and enter the following expression:
centroid( $geometry )
It is worth noting that the correct geometry type has to be set manually. If a wrong type is set, the symbol layer can not be rendered.
Drawing buffers around features
Buffers are an example of a polygon geometry generator layer. The second parameter of the buffer function defines if the buffer is generated outside (for positive values) or inside (for negative values) of the feature. The value has to be provided in the layer’s CRS units, in this case, that means an inner buffer of 0.005 degrees:
buffer( $geometry, -0.005 )
Creating a line between features in different layers
The following expression creates lines from all district centroids (as shown in the first example) and a feature from the Citybike layer where the STATION attribute value is ‘Millennium Tower’:
make_line( centroid( $geometry ), geometry( get_feature( 'Citybike', 'STATION', 'Millennium Tower' ) ) )
More advanced examples
Using these basic examples as a starting point, geometry generators open a wide field of advanced symbology options. For example, this sector light style presented on GIS.Stackexchange or my recently introduced conveyor belt flow style:
Sometimes I prefer to publish my map in gray instead of black. But all newly added QGIS composer items are set to black by default. For changing the colors more easily and rapidly I created the “Turn Gray” plugin. By default it changes all foreground colors (labels and outlines) to gray. But you are free … Continue reading Gray is the new Black
It’s been a long time since I last blogged here. Let’s just blame that on the amount of changes going into QGIS 3.0 and move on…
One new feature which landed in QGIS 3.0 today is a processing algorithm for automatic coloring of a map in such a way that adjoining polygons are all assigned different color indexes. Astute readers may be aware that this was possible in earlier versions of QGIS through the use of either the (QGIS 1.x only!) Topocolor plugin, or the Coloring a map plugin (2.x).
What’s interesting about this new processing algorithm is that it introduces several refinements for cartographically optimising the coloring. The earlier plugins both operated by pure “graph” coloring techniques. What this means is that first a graph consisting of each set of adjoining features is generated. Then, based purely on this abstract graph, the coloring algorithms are applied to optimise the solution so that connected graph nodes are assigned different colors, whilst keeping the total number of colors required minimised.
The new QGIS algorithm works in a different way. Whilst the first step is still calculating the graph of adjoining features (now super-fast due to use of spatial indexes and prepared geometry intersection tests!), the colors for the graph are assigned while considering the spatial arrangement of all features. It’s gone from a purely abstract mathematical solution to a context-sensitive cartographic solution.
Let’s explore the differences. First up, the algorithm has an option for the “minimum distance between features”. It’s often the case that features aren’t really touching, but are instead just very close to each other. Even though they aren’t touching, we still don’t want these features to be assigned the same color. This option allows you to control the minimum distance which two features can be to each other before they can be assigned the same color.
The biggest change comes in the “balancing” techniques available in the new algorithm. By default, the algorithm now tries to assign colors in such a way that the total number of features assigned each color is equalised. This avoids having a color which is only assigned to a couple of features in a large dataset, resulting in an odd looking map coloration.
Another available balancing technique is to balance the color assignment by total area. This technique assigns colors so that the total area of the features assigned to each color is balanced. This mode can be useful to help avoid large features resulting in one of the colors appearing more dominant on a colored map.
The final technique, and my personal preference, is to balance colors by distance between colors. This mode will assign colors in order to maximize the distance between features of the same color. Maximising the distance helps to create a more uniform distribution of colors across a map, and avoids certain colors clustering in a particular area of the map. It’s my preference as it creates a really nice balanced map – at a glance the colors look “randomly” assigned with no discernible pattern to the arrangement.
As these examples show, considering the geographic arrangement of features while coloring allows us to optimise the assigned colors for cartographic output.
The other nice thing about having this feature implemented as a processing algorithm is that unlike standalone plugins, processing algorithms can be incorporated as just one step of a larger model (and also reused by other plugins!).
QGIS 3.0 has tons of great new features, speed boosts and stability bumps. This is just a tiny taste of the handy new features which will be available when 3.0 is released!
Today’s post was motivated by a question following up on my recent post “Details of good flow maps“: How to create arrows with gradients from transparent to opaque?
The key idea is to use a gradient fill to color the arrows:
It all seems perfectly straightforward: determine the direction of the line and set the gradient rotation according to the line direction.
But wait! That doesn’t work!
The issue is that all default angle functions available in expressions return clockwise angles but the gradient rotation has to be set in counter-clockwise angles. So we need this expression:
Sorry, this entry is only available in the Dutch language
In my previous posts, I discussed classic flow maps that use arrows of different width to encode flows between regions. This post presents an alternative take on visualizing flows, without any arrows. This style is inspired by Go with the Flow by Robert Radburn and Visualisation of origins, destinations and flows with OD maps by J. Wood et al.
The starting point of this visualization is a classic OD matrix.
For my previous flow maps, I already converted this data into a more GIS-friendly format: a Geopackage with lines and information about the origin, destination and strength of the flow:
In addition, I grabbed state polygons from Natural Earth Data.
At this point, we have 72 flow features and 9 state polygon features. An ordinary join in the layer properties won’t do the trick. We’d still be stuck with only 9 polygons.
Virtual layers to the rescue!
The QGIS virtual layers feature (Layer menu | Add Layer | Add/Edit Virtual Layer) provides database capabilities without us having to actually set up a database … *win!*
Using a classic SQL query, we can join state polygons and migration flows into a new virtual layer:
The resulting virtual layer contains 72 polygon features. There are 8 copies of each state.
Now that the data is ready, we can start designing the visualization in the Print Composer.
This is probably the most manual step in this whole process: We need 9 map items, one for each mini map in the small multiples visualization. Create one and configure it to your liking, then copy and paste to create 8 more copies.
I’ve decided to arrange the map items in a way that resembles the actual geographic location of the state that is represented by the respective map, from the state of Vorarlberg (a proud QGIS sponsor by the way) in the south-west to Lower Austria in the north-east.
To configure which map item will represent the flows from which origin state, we set the map item ID to the corresponding state ID. As you can see, the map items are numbered from 1 to 9:
Once all map items are set up, we can use the map item IDs to filter the features in each map. This can be implemented using a rule based renderer:
The first rule will ensure that the each map only shows flows originating from a specific state and the second rule will select the state itself.
We configure the symbol of the first rule to visualize the flow strength. The color represents the number number of people moving to the respective district. I’ve decided to use a smooth gradient instead of predefined classes for the polygon fill colors. The following expression maps the feature’s weight value to a shade on the Viridis color ramp:
ramp_color( 'Viridis', scale_linear("weight",0,2000,0,1) )
You can use any color ramp you like. If you want to use the Viridis color ramp, save the following code into an .xml file and import it using the Style Manager. (This color ramp has been provided by Richard Styron on rocksandwater.net.)
<!DOCTYPE qgis_style> <qgis_style version="0"> <symbols/> <colorramp type="gradient" name="Viridis"> <prop k="color1" v="68,1,84,255"/> <prop k="color2" v="253,231,36,255"/> <prop k="stops" v="0.04;71,15,98,255:0.08;72,29,111,255:0.12;71,42,121,255:0.16;69,54,129,255:0.20;65,66,134,255:0.23;60,77,138,255:0.27;55,88,140,255:0.31;50,98,141,255:0.35;46,108,142,255:0.39;42,118,142,255:0.43;38,127,142,255:0.47;35,137,141,255:0.51;31,146,140,255:0.55;30,155,137,255:0.59;32,165,133,255:0.62;40,174,127,255:0.66;53,183,120,255:0.70;69,191,111,255:0.74;89,199,100,255:0.78;112,206,86,255:0.82;136,213,71,255:0.86;162,218,55,255:0.90;189,222,38,255:0.94;215,226,25,255:0.98;241,229,28,255"/> </colorramp> </colorramps> </qgis_style>
If we go back to the Print Composer and update the map item previews, we see it all come together:
Finally, we set title, legend, explanatory texts, and background color:
I think it is amazing that we are able to design a visualization like this without having to create any intermediate files or having to write custom code. Whenever a value is edited in the original migration dataset, the change is immediately reflected in the small multiples.
A year ago I have asked QGIS’s community what were their favourite QGIS new features from 2015 and published this blog post. This year I decided to ask it again. In 2016, we add the release of the second long-term release (2.14 LTR), and two other stable versions (2.16 and 2.18).
2016 was again very productive year for the QGIS community, with lots of improvements and new features landing on QGIS source code, not to speak of all the work already in place for QGIS 3. This is a great assurance of the project’s vitality.
5 – Paste a style to multiple selected layers or to all layers in a legend group (2.14)
This is a productivity functionaly that I just realized that existed now, with so many people voting on it. If copy/paste styles was, in my opinion, a killer feature, being able to use it in multiple layers or even a group is just great.
4 – fTools plugin has been replaced with Processing algorithms (2.16)
While checking the Vector Menu, the tools seem the same as previous version, but it’s when you open them that you understand the difference. All vector tools, provided until now by the fTools core plugin, were replaced by equivalent processing Algoritms. For the users it means easier access to more functionality, like running the tools in batch mode, or getting outputs as temporary layers. Besides some of the tools have been improved.
3 – Virtual layers (2.14)
This is definitly one of my favourite new features, and it seems I’m not alone. With virtual layers you can run SQL queries using the layers loaded in the project, even when the layers are not stored in a relational database. We are not talking about WHERE statments to filter data, with this you can do real SQL queries, with spatial analysis, aggregations, and so on. Besides, virtual layers will act as VIEWs and any changes to any of the input layers will automatically update the layer.
2 – Speed and memory improvements (2.14)
It’s no surprise that speed and memory improvements we one of the most voted features. Lots of improvements were made for loading and managing large datasets, and this have a tremendous impact in all users. According to the changelog, zoom is faster, selecting features is faster, updating attributes on selected features is faster, and it consumes less memory. So don’t be afraid to put QGIS to the test.
1 – Trace digitising tool (2.14)
If you do lots of digitising, you better look into this new feaure that landed on QGIS 2.14. It allows you to digitize new feature by using other layers boundaries. Besides the quality improvement of layers topology, this can make digitizing almost feel pleasing and fast! Just click the first point, move your mouse around other features edged to pick up more vertex.
There were other new features that also made the delight of many users. For example, several improvements on the labeling, Georeference outputs (eg PDF) from composer (2.16), Filter legend by expression (2.14), 2.5D Renderer. Personally, the Style docker made my day/year. But you can check the full results of the survey, if you like.
Obviously, this list means nothing at all. All new features were of tremendous value, and will be useful for thousands (yes thousands) of people. It was a mere exercise as, with such a diverse QGIS crowd, it would be impossible to build a list that would fit us all. Besides, there were many great enhancements, introduced during 2016, that might have fallen under the radar for most users. Check the visual changelogs for a full list of new features.
On my behalf, to all developers, sponsors, and general QGIS contributors, once again
THANK YOU VERY MUCH FOR YOUR TREMENDOUS WORK!
I wish you a fantastic 2017.
QGIS Developer Sprint in Lyon
QGIS Server 3.0 is going to be better than ever! Last week I attended to the mini code-sprint organized by the french QGIS developers in Lyon.
The code sprint was focused on QGIS Server refactoring to reach the following goals:
- increase maintainability through modularity and clean code responsibilities
- increase performances
- better multi-project handling and caching
- multi threaded rendering
By working for different companies on such a big Open Source project like QGIS, coordination between developers is fundamentally achieved through those kind of events.
We were a small group of engaged QGIS Server developers and I think that the alternance between brainstorming and coding has proven to be very productive: after two days we were able to set common milestones and commitments that will ensure a bright future to QGIS Server.
A huge and warm thank to the french QGIS developers that organized this meeting!
Last time, I wrote about the little details that make a good flow map. The data in that post was made up and simpler than your typical flow map. That’s why I wanted to redo it with real-world data. In this post, I’m using domestic migration data of Austria.
With 9 states, that makes 72 potential flow arrows. Since that’s too much to map, I’ve decided in a first step to only show flows with more than 1,000 people.
Following the recommendations mentioned in the previous post, I first designed a basic flow map where each flow direction is rendered as a black arrow:
Even with this very limited number of flows, the map gets pretty crowded, particularly around the north-eastern node, the Austrian capital Vienna.
To reduce the number of incoming and outgoing lines at each node, I therefore decided to change to colored one-sided arrows that share a common geometry:
The arrow color is determined automatically based on the arrow direction using the following expression:
CASE WHEN "weight" < 1000 THEN color_rgba( 0,0,0,0) WHEN x(start_point( $geometry)) - x(end_point($geometry)) < 0 THEN '#1f78b4' ELSE '#ff7f00' END
The same approach is used to control the side of the one-sided arrow head. The arrow symbol layer has two “arrow type” options for rendering the arrow head: on the inside of the curve or on the outside. This means that, if we wouldn’t use a data-defined approach, the arrow head would be on the same side – independent of the line geometry direction.
CASE WHEN x(start_point( $geometry)) - x(end_point($geometry)) < 0 THEN 1 ELSE 2 END
Obviously, this ignores the corner case of start and end points at the same x coordinate but, if necessary, this case can be added easily.
Of course the results are far from perfect and this approach still requires manual tweaking of the arrow geometries. Nonetheless, I think it’s very interesting to see how far we can push the limits of data-driven styling for flow maps.
Give it a try! You’ll find the symbol and accompanying sample data on the QGIS resource sharing plugin platform:
In my previous post, I shared a flow map style that was inspired by a hand drawn map. Today’s post is inspired by a recent academic paper recommended to me by Radoslaw Panczak @RPanczak and Thomas Gratier @ThomasG77:
Jenny, B., Stephen, D. M., Muehlenhaus, I., Marston, B. E., Sharma, R., Zhang, E., & Jenny, H. (2016). Design principles for origin-destination flow maps. Cartography and Geographic Information Science, 1-15.
Jenny et al. (2016) performed a study on how to best design flow maps. The resulting design principles are:
- number of flow overlaps should be minimized;
- sharp bends and excessively asymmetric flows should be avoided;
- acute intersection angles should be avoided;
- flows must not pass under unconnected nodes;
- flows should be radially arranged around nodes;
- quantity is best represented by scaled flow width;
- flow direction is best indicated with arrowheads;
- arrowheads should be scaled with flow width, but arrowheads for thin flows should be enlarged; and
- overlaps between arrowheads and flows should be avoided.
Many of these points concern the arrangement of flow lines but I want to talk about those design principles that can be implemented in a QGIS line style. I’ve summarized the three core ideas:
- use arrow heads and scale arrow width according to flow,
- enlarge arrow heads for thin flows, and
- use nodes to arrange flows and avoid overlaps of arrow heads and flows
To get started, we can use a standard QGIS arrow symbol layer. To represent the flow value (“weight”) according to the first design principle, all arrow parameters are data-defined:
To enlarge the arrow heads for thin flow lines, as required by the second design principle, we can add a fixed value to the data-defined head length and thickness:
The main issue with this flow map is that it gets messy as soon as multiple arrows end at the same location. The arrow heads are plotted on top of each other and at some point it is almost impossible to see which arrow starts where. This is where the third design principle comes into play!
To fix the overlap issue, we can add big round nodes at the flow start and end points. These node buffers are both used to render circles on the map, as well as to shorten the arrows by cutting off a short section at the beginning and end of the lines:
difference( difference( $geometry, buffer( start_point($geometry), 10000 ) ), buffer( end_point( $geometry), 10000 ) )
Note that the buffer values in this expression only produce appropriate results for line datasets which use a CRS in meters and will have to be adjusted for other units.
It’s great to have some tried and evaluated design guidelines for our flow maps. As always: Know your cartography rules before you start breaking them!
PS: To draw a curved arrow, the line needs to have one intermediate point between start and end – so three points in total. Depending on the intermediate point’s position, the line is more or less curved.
The QGIS map style I want to share with you today was inspired by a hand-drawn map by Philippe Rekacewicz that I saw on Twitter:
The look reminds me of conveyor belts, thus the name choice.
The conveyor belt is a line symbol that makes extensive use of Geometry generators. One generator for the circle at the flow line start and end point, respectively, another generator for the belt, and a final one for the small arrows around the colored circles. The color and size of the circle are data defined:
The collection also contains a sample Geopackage dataset which you can use to test the symbol immediately. It is worth noting that the circle size has to be specified in layer CRS units.
It’s great fun playing with the power of Geometry generator symbol layers and QGIS geometry expressions. For example, this is the expression for the final geometry that is used to draw the small arrows around colored circles:
line_merge( intersection( exterior_ring( convex_hull( union( buffer( start_point($geometry), "start_size" ), buffer( end_point($geometry), 500000 ) ) ) ), exterior_ring( buffer( start_point( $geometry), "start_size" ) ) ) )
The expression constructs buffer circles, the belt geometry (convex_hull around buffers), and finally extracts the intersecting part from the start circle and the belt geometry.
Hope you enjoy it!
It’s holiday season, why not share one of your own symbols with the QGIS community?
QGIS’ handling of color ramps has just gotten much better with a series of improvements we committed to the open source project’s upcoming version 3.0.
This slide goes through brief summary of changes: Color ramp handling, made fun!
On the developer front, one nice improvement is the addition of an invert() function directly attached to color ramp classes (QgsColorRamp and its children). This removed the need for symbol layers and renderers to implement individual invert-related functions; those are now served with a customized source color ramp, with edited steps and/or reversed order already taken into account.
The following packages can now be installed:
- qgis 2.18.0
- qgis-debuginfo 2.18.0
- qgis-devel 2.18.0
- qgis-grass 2.18.0
- qgis-python 2.18.0
- qgis-server 2.18.0
Installation instructions (run as “root” user or use “sudo”):
su # Fedora 23, Fedora 24: dnf copr enable neteler/QGIS-2.18-Las-Palmas dnf update # note: the "qca-ossl" package is the OpenSSL plugin for QCA dnf install qgis qgis-grass qgis-python qca-ossl
In the previous post, I presented an approach to generalize big trajectory datasets by extracting flows between cells of a data-driven irregular grid. This generalization provides a much better overview of the flow and directionality than a simple plot of the original raw trajectory data can. The paper introducing this method also contains more advanced visualizations that show cell statistics, such as the overall count of trajectories or the generalization quality. Another bit of information that is often of interest when exploring movement data, is the time of the movement. For example, at LBS2016 last week, M. Jahnke presented an application that allows users to explore the number of taxi pickups and dropoffs at certain locations:
By adopting this approach for the generalized flow maps, we can, for example, explore which parts of the research area are busy at which time of the day. Here I have divided the day into four quarters: night from 0 to 6 (light blue), morning from 6 to 12 (orange), afternoon from 12 to 18 (red), and evening from 18 to 24 (dark blue).
The resulting visualization shows that overall, there is less movement during the night hours from midnight to 6 in the morning (light blue quarter). Sounds reasonable!
One implementation detail worth considering is which timestamp should be used for counting the number of movements. Should it be the time of the first trajectory point entering a cell, or the time when the trajectory leaves the cell, or some average value? In the current implementation, I have opted for the entry time. This means that if the tracked person spends a long time within a cell (e.g. at the work location) the trip home only adds to the evening trip count of the neighboring cell along the trajectory.
Since the time information stored in a PostGIS LinestringM feature’s m-value does not contain any time zone information, we also have to pay attention to handle any necessary offsets. For example, the GeoLife documentation states that all timestamps are provided in GMT while Beijing is in the GMT+8 time zone. This offset has to be accounted for in the analysis script, otherwise the counts per time of day will be all over the place.
Using the same approach, we could also investigate other variations, e.g. over different days of the week, seasonal variations, or the development over multiple years.
In the fist two parts of the Movement Data in GIS series, I discussed modeling trajectories as LinestringM features in PostGIS to overcome some common issues of movement data in GIS and presented a way to efficiently render speed changes along a trajectory in QGIS without having to split the trajectory into shorter segments.
While visualizing individual trajectories is important, the real challenge is trying to visualize massive trajectory datasets in a way that enables further analysis. The out-of-the-box functionality of GIS is painfully limited. Except for some transparency and heatmap approaches, there is not much that can be done to help interpret “hairballs” of trajectories. Luckily researchers in visual analytics have already put considerable effort into finding solutions for this visualization challenge. The approach I want to talk about today is by Andrienko, N., & Andrienko, G. (2011). Spatial generalization and aggregation of massive movement data. IEEE Transactions on visualization and computer graphics, 17(2), 205-219. and consists of the following main steps:
- Extracting characteristic points from the trajectories
- Grouping the extracted points by spatial proximity
- Computing group centroids and corresponding Voronoi cells
- Deviding trajectories into segments according to the Voronoi cells
- Counting transitions from one cell to another
The authors do a great job at describing the concepts and algorithms, which made it relatively straightforward to implement them in QGIS Processing. So far, I’ve implemented the basic logic but the paper contains further suggestions for improvements. This was also my first pyQGIS project that makes use of the measurement value support in the new geometry engine. The time information stored in the m-values is used to detect stop points, which – together with start, end, and turning points – make up the characteristic points of a trajectory.
The following animation illustrates the current state of the implementation: First the “hairball” of trajectories is rendered. Then we extract the characteristic points and group them by proximity. The big black dots are the resulting group centroids. From there, I skipped the Voronoi cells and directly counted transitions from “nearest to centroid A” to “nearest to centroid B”.
The resulting visualization makes it possible to analyze flow strength as well as directionality. I have deliberately excluded all connections with a count below 10 transitions to reduce visual clutter. The cell size / distance between point groups – and therefore the level-of-detail – is one of the input parameters. In my example, I used a target cell size of approximately 2km. This setting results in connections which follow the major roads outside the city center very well. In the city center, where the road grid is tighter, trajectories on different roads mix and the connections are less clear.
Since trajectories in this dataset are not limited to car trips, it is expected to find additional movement that is not restricted to the road network. This is particularly noticeable in the dense area in the west where many slow trajectories – most likely from walking trips – are located. The paper also covers how to ensure that connections are limited to neighboring cells by densifying the trajectories before computing step 4.
Running the scripts for over 18,000 trajectories requires patience. It would be worth evaluating if the first three steps can be run with only a subsample of the data without impacting the results in a negative way.
One thing I’m not satisfied with yet is the way to specify the target cell size. While it’s possible to measure ellipsoidal distances in meters using QgsDistanceArea (irrespective of the trajectory layer’s CRS), the initial regular grid used in step 2 in order to group the extracted points has to be specified in the trajectory layer’s CRS units – quite likely degrees. Instead, it may be best to transform everything into an equidistant projection before running any calculations.
It’s good to see that PyQGIS enables us to use the information encoded in PostGIS LinestringM features to perform spatio-temporal analysis. However, working with m or z values involves a lot of v2 geometry classes which work slightly differently than their v1 counterparts. It certainly takes some getting used to. This situation might get cleaned up as part of the QGIS 3 API refactoring effort. If you can, please support work on QGIS 3. Now is the time to shape the PyQGIS API for the following years!
The possibility to easily share plugins with other users and discover plugins written by other community members has been a powerful feature of QGIS for many years.
The QGIS Resources Sharing plugin is meant to enable the same sharing for map design resources. It allows you to share collections of resources, including but not limited to SVGs, symbols, styles, color ramps, and processing scripts.
Using the Resource Sharing plugin is like using the Plugin Manager. Once installed, you are presented with a list of available resource collections for download. You will find that there are already some really nice collections, including nautical symbols, Mapbox Maki Icons, and my Google-like OSM road style.
By pressing Install, the resource collection is downloaded and you can have a look at the content using the Open folder button. In case of the Mapbox Maki Icon collection, it contains a folder of SVGs:
Using the new icons is as simple as opening the layer styling settings and selecting the Mapbox Maki Icons collection in the SVG group list:
Similarly, if you download the OSM Spatialite Googlemaps collection, its road line symbols are added to your existing list of available line symbols:
By pressing the Open Library button, you get to the Style Manager where you can browse through all installed symbols and delete, rename, or categorize them.
The Resource Sharing plugin was developed by Akbar Gumbira during this year’s Google Summer of Code. The full documentation, including instructions for how to share your own symbols with the community, is available at www.akbargumbira.com/qgis_resources_sharing.