QGIS Planet

WMTS Enhancement and XYZ Tile Native Support in QGIS 2.18

In this post, we will highlight the new features we have added to QGIS 2.18 …

WMTS enhancement

The WMTS provider had not been benefiting from the the QGIS multi-threaded rendering we did earlier in QGIS 2.4.

In previous versions of QGIS, users had to wait until download of all tiles of a layer has finished in order to view the resulting map. This has now been fixed and the tiles show up in map canvas immediately as they get downloaded, improving the user experience by greatly lowering the time until something is shown.

Moreover, previously downloaded tiles from lower or higher resolutions may be used for the preview functionality in the areas where the tiles with correct resolution have not been downloaded yet.

The screencast below shows fetching and rendering a WMTS layer in QGIS 2.14 (left) and the same layer in QGIS 2.18 (right):

Support for XYZ raster tiles

There are a couple of python plugins allowing users to add XYZ tiles (e.g. Bing maps) to QGIS. The plugins only allow certain web services and it is often tricky for supporting the private ones with API keys.

In addition, there are other QGIS applications without python support (e.g. QGIS for Android devices) where they can leverage from having a native support.

Currently, you can only add XYZ tile services from the Browser panel. The video below demonstrates how to add the current precipitation and OpenStreetMap xyz tiles to your QGIS:

Sponsors

WMTS enhancements was sponsored by Land Information New Zealand. Support for XYZ tiles was funded internally.

Editing Raster Cell Values in QGIS Using Serval Plugin

Users can directly edit raster cell values using Serval plugin in QGIS.

Read more for how to use this plugin…

How to use Serval

Serval is available from QGIS plugin repository. Note that you will need to restart QGIS if you upgrade Serval from an earlier version.

Once installed, Serval functions and settings will be available from the toolbar.

Serval Toolbar in QGIS

Serval supports Undo/Redo for editing values of raster. But it is recommended to make a copy of your raster.

Currently, the following functionalities are available:

  • Probe mode Displays raster bands values in boxes.
  • Draw mode Draw/Edit mode: bands values can be modified in the boxes and written to the current raster cell by hitting the Enter key. In this mode the values will be also assigned to any other raster cell clicked by user.
  • Write nodata To replace a cell value with the NODATA value.
  • Define nodata To define or replace the NODATA value.
  • Color picker To pick a color using QGIS color picker (3-bands rasters only).
  • Undo Redo To Undo/Redo the cell edit. Edits history is saved separately for each raster, that is, undo/redo is always done for current raster layer.

Future developments

We’d like to add support to edit values using spatial and expression selection tools.

For any problems or feedback, please consider to file a ticket here.

Crayfish 2.3: New Features

Crayfish 2.3 is out with FLO-2D format support and automatic export of contours

Here are the new features in more detail…

Support for new formats

We added support for FLO-2D result files.

FLO-2D format in Crayfish

Weather Research and Forecasting Model (WRF) outputs are now fully supported in Crayfish.

Export contours

Users can now directly generate vector contours from a Crayfish layer. The export-to-contour feature allows you to select type of contour (e.g. line or area contour) and contour intervals. A very handy option is that the current colour ramp can be also used for your contour intervals.

Exporting contours in Crayfish

Processing toolbox Crayfish provider

We have incorporated many algorithms from Crayfish to processing toolbox (Export grid, Export mesh elements, …). To activate the Crayfish module, from the main menu, select Processing > Options and in the new window, under Providers > Crayfish algorithms select the option for Activate.

Exporting contours in Crayfish

With the processing toolbox, user can create batch geo-processing algorithms to automate their work. For example, you can create a batch process using this module, to export several Crayfish grids to rasters.

Export to animation using the Processing Toolbox is not supported yet.

Future developments

We’d like to add support for mesh generation and also mesh calculator. If these are something of you or your organisation interest and would like to contribute financially, feel free to get in touch.

For any problems or feedback, please consider to file a ticket here.

Introducing First Aid Plugin

First Aid Plugin

Software development often consists of a brief period of pure excitement while writing the bulk of the source code, followed by a long and dreaded period of finding bugs and fixing them. This scheme is no different when writing plugins for QGIS.

Because we also write QGIS plugins ourselves, we were thinking of how to ease the pain of getting rid of bugs — and so we ended up creating the First Aid plugin, now available in the official QGIS plugin repository.

It is meant to be a Swiss army knife for QGIS plugin developers, a tool that allows easy inspection of any Python code running within QGIS. This is important because it can potentially save developers a lot of their valuable time.

How many times did you end up adding “print” statements into the code to find out what was going wrong in your code? With First Aid plugin this should be no longer necessary.

Error Handler

So let me explain what does it do. First of all, it comes with an improved Python error handler. What this means is that whenever an error occurs in code of a plugin, a window with all the details comes up.

Previously in QGIS you could only find out what was the exception’s type, message and stack trace. First Aid plugin adds source code view, variables view and even an embedded Python console where you can further inspect the state of the plugin at the time of the error. Here is how it looks like in action:

Error Handler Screenshot

Debugger

Plugins however do not always come up with errors that can be caught and handled by QGIS. More often plugins simply do not behave as one would expect them to. Here is when people usually resort to using debuggers. There are IDEs like PyDev or PyCharm – or even standalone tools like Winpdb – that allow developers to do remote debugging. Basically they can connect to the Python environment within QGIS and debug the code there. Personally, I have never been a big fan of this approach and found remote debugging cumbersome to set up and use. And trying to debug something on a client’s computer is even a greater challenge.

First Aid fortunately integrates a debugger into QGIS environment. This allows developers to simply open the debugger window, load some Python files, set breakpoints and everything is ready. Once QGIS reaches a line with a breakpoint, the debugger window will be activated and it is possible to step through the code and inspect the variables to understand what is going on in the code.

Debugger Screenshot

The great thing is that once the execution of Python code is stopped, it is possible to step into code, step over, step out or run to cursor, just like in any other debugger. It is also possible to run custom scripts from within debugger window — they will be also run in debug mode.

Debugging is active only while the debugger window is still open. While debugging, there is some extra overhead when running any Python code (even for code that you do not intend to debug), so it is better to close the debugger when not needed.

The plugin has already helped us various times to quickly identify problems in plugins. Having said that, please note that the plugin is still quite young and may not work perfectly in all cases. We would be happy to hear your feedback. Any issues or pull requests on GitHub would be greatly appreciated!

QGIS Code Sprint - FOSS4G UK - 2016

Issue Reporting

The Open Source community in the UK will get together for another great FOSS4G meeting from 14 to 16 of June in Southampton.

QGIS has already established itself as one of the major Open Source GIS applications. To help the community to contribute back to the project, we are running a code sprint.

The code sprint session will not require prior knowledge of coding, so all participants are welcome to join. Make sure you bring your laptop!

There will be several power users and plugin developers to help with the code sprint. There will be 3 topics covered during the code sprint:

  • Documentation
  • Helping with the bug fixing
  • Contributing to qgis2web plugin

Documentation

If you are interested in helping with the QGIS documentation, we will run a short presentation to introduce you to the workflow.

Participants are required to have installed and configured the following software packages:

Helping with the bug fixing

For those who are interested in findings bugs, testing existing bugs and leaning up the bug queue, you will need:

  • An OSGeo ID
    • Currently requires contacting OSGeo admins on IRC (#osgeo) in order to get the registration form (!)
  • Multiple versions of QGIS
    • QGIS LTR (LTS)
    • QGIS release
    • QGIS master

If you are familiar with coding (in C++ and python) and would like to fix some bugs, you will need:

  • QGIS development environment
  • Github account

Contributing to qgis2web plugin

qgis2web

Help improve qgis2web by joining any of these activities:

What you’ll need

  • laptop
  • QGIS, ideally current or nightly
  • a Github account
  • whatever Git/Github client you prefer
  • your favourite text editor

Details of some specific bugs and features are listed in a dedicated Github issue. If you are not coming to FOSS4GUK 2016, we’d still really value any help you can give. We’ll keep an eye on Github, or try the qgis2web Gitter.

This is a great opportunity to take your first steps in contributing to an open-source project. Come and help – we’ll tell you everything you need to know, and help you learn the skills you don’t yet have. Rest assured that we will be at least as grateful as you for the help you give.

If you are a beginner, @archaeogeek is also running a great workshop at FOSS4GUK 2016 on getting started in open-source: Don’t be afraid to commit

Using OS AddressBase for Address Search in QGIS

In this blog post we’ll learn how to use Ordnance Survey AddressBase data with the QGIS Discovery plugin for searching addresses.

Discovery Plugin for QGIS

Before we start

The AddressBase data will be loaded into a PostGIS table for Discovery to query. At this stage we should already have a functioning PostgreSQL / PostGIS installation.

A previous blog post describes how to quickly set up such an environment.

Creating the addressbase table

Let’s now create a table for storing the addressbase data. In the example below we’ll create a table called addressbase in the os_address schema.

The script below can be executed through pgAdminIII.

To run the script:

  1. Open pgAdminIII
  2. Connect to your destination database
  3. Select Query tool from the Tools menu
  4. Paste the code below into the Query tool
  5. Press F5 to execute the query (it may take a few seconds to complete)

When the query has finished you should see Query returned successfully with no result in … seconds. in the Messages panel:

pgAdminIII Messages Panel

At this point we should be able to locate the new addressbase table within the os_address schema:

addressbase Table

If you can’t see the schema / table you probably need to refresh the schemas / tables views in pgAdminIII’s Object browser panel by hitting F5.

``` — Create the destination schema if required CREATE SCHEMA IF NOT EXISTS os_address;

— Create a function which will populate the full_address and geom columns as — data are imported CREATE OR REPLACE FUNCTION create_geom_and_address() RETURNS trigger AS $$ BEGIN — The geometry — Set it based on the x_coord and y_coord fields NEW.geom = ST_SetSRID(ST_MakePoint(NEW.x_coordinate, NEW.y_coordinate), 27700); — The full address — Initialise it NEW.full_address = ‘’; — Build the full address by only including optional address components if they — exist IF NEW.organisation_name IS NOT NULL AND length(NEW.organisation_name) > 0 THEN

NEW.full_address = NEW.full_address || NEW.organisation_name || ', ';

END IF; IF NEW.department_name IS NOT NULL AND length(NEW.department_name) > 0 THEN

NEW.full_address = NEW.full_address || NEW.department_name || ', ';

END IF; IF NEW.po_box_number IS NOT NULL AND length(NEW.po_box_number) > 0 THEN

NEW.full_address = NEW.full_address || NEW.po_box_number || ', ';

END IF; IF NEW.sub_building_name IS NOT NULL AND length(NEW.sub_building_name) > 0 THEN

NEW.full_address = NEW.full_address || NEW.sub_building_name || ', ';

END IF; IF NEW.building_name IS NOT NULL AND length(NEW.building_name) > 0 THEN

NEW.full_address = NEW.full_address || NEW.building_name || ', ';

END IF; IF NEW.building_number IS NOT NULL THEN

NEW.full_address = NEW.full_address || NEW.building_number || ', ';

END IF; IF NEW.dependent_thoroughfare IS NOT NULL AND length(NEW.dependent_thoroughfare) > 0 THEN

NEW.full_address = NEW.full_address || NEW.dependent_thoroughfare || ', ';

END IF; IF NEW.thoroughfare IS NOT NULL AND length(NEW.thoroughfare) > 0 THEN

NEW.full_address = NEW.full_address || NEW.thoroughfare || ', ';

END IF;

NEW.full_address = NEW.full_address || NEW.post_town || ‘, ’;

IF NEW.double_dependent_locality IS NOT NULL AND length(NEW.double_dependent_locality) > 0 THEN

NEW.full_address = NEW.full_address || NEW.double_dependent_locality || ', ';

END IF; IF NEW.dependent_locality IS NOT NULL AND length(NEW.dependent_locality) > 0 THEN

NEW.full_address = NEW.full_address || NEW.dependent_locality || ', ';

END IF;

NEW.full_address = NEW.full_address || NEW.postcode;

RETURN NEW; END; $$ LANGUAGE ‘plpgsql’;

— Drop any existing addressbase table DROP TABLE IF EXISTS os_address.addressbase CASCADE; CREATE TABLE os_address.addressbase ( — id will be the primary key, populated automatically id serial NOT NULL, uprn bigint NOT NULL, os_address_toid varchar(24) NOT NULL, — os_address_toid bigint NOT NULL, udprn integer NOT NULL, organisation_name varchar(60), department_name varchar(60), po_box_number varchar(6), sub_building_name varchar(30), building_name varchar(50), building_number smallint, dependent_thoroughfare varchar(80), thoroughfare varchar(80), post_town varchar(30) NOT NULL, double_dependent_locality varchar(35), dependent_locality varchar(35), postcode varchar(8) NOT NULL, postcode_type char(1) NOT NULL, x_coordinate numeric(8,2) NOT NULL, y_coordinate numeric(9,2) NOT NULL, latitude numeric(9,7) NOT NULL, longitude numeric(8,7) NOT NULL, rpc char(1) NOT NULL, country char(1) NOT NULL, change_type char(1) NOT NULL, la_start_date date NOT NULL, rm_start_date date NOT NULL, last_update_date date NOT NULL, class char(1) NOT NULL, — the next two fields are populated automatically on insert full_address text NOT NULL, geom geometry(Point,27700) NOT NULL, CONSTRAINT addressbase_pkey PRIMARY KEY (id) ) WITH ( OIDS=FALSE );

— Create a pg_trgm index on the full_address column — This will allow super-fast, case-insensitive search on the column CREATE EXTENSION IF NOT EXISTS pg_trgm; CREATE INDEX addressbase_full_address_gin_trgm ON os_address.addressbase USING gin (“full_address” gin_trgm_ops);

— Spatial index for the geometry column CREATE INDEX addressbase_geom_gist ON os_address.addressbase USING gist (geom);

— trigger to create points and addresses — This trigger will be executed on each row inserted, calling the function defined above CREATE TRIGGER tr_create_geom_and_address BEFORE INSERT ON os_address.addressbase FOR EACH ROW EXECUTE PROCEDURE create_geom_and_address();

```

The script above has:

  • Created a table
  • Added any necessary indices
  • Created two additional, derived columns, full_address and geom

full_address will be used to store various address components into a sensible, human readable address. geom will be used to store point geometry based on address eastings/northings.

See the script comments for more information / detail.

Loading AddressBase

At this point we have an empty table ready to accept our AddressBase data. We will now import the data using pgAdminIII. Extract the CSV files for the addresses, you should end up seeing one or more CSV files, for example AddressBase_FULL_2016-03-19_001.csv

In pgAdminIII:

  1. Locate the addressbase table
  2. Right click it, select Import

An import dialog should appear. Select the first CSV file and set the settings in the File Options tab as shown here:

AddressBase Import Options 1

Uncheck the id, full_address and geom columns in the Columns tab as shown here:

AddressBase Import Options 2

Click Import. After a few seconds the dialog may report (Not Responding). This is nothing to worry about, be patient.

When the import process completes, close the import dialog and repeat the above steps with any remaining CSV files.

At this stage the data has been imported and the full_address field should contain sensible, human-readable addresses.

Configuring Discovery

With the data loaded in QGIS, we can now configure Discovery to make use of it.

  1. Install the Discovery plugin if not already installed
  2. Open Discovery’s settings using the button
  3. Set the settings as follows, changing the Scale Expression if required

Discovery Settings for OS AddressBase

Congratulations! QGIS should now be set up to search your AddressBase data.

QGIS Report Plugin Release

Issue Reporting

We develop several public and private plugins for QGIS users and our clients. During the testing phase, they often come across some bugs. To enable them to report the issues and include the right type of information, we have developed The Report plugin to simplify the process Report plugin.

The Report plugin helps developers get better bug reports from users, by auto-populating the relevant information. It also simplifies the bug reporting process for users.

The Report plugin can automatically extract information from the plugin crash as well as other important information for developers. With just one click you can report the issue to the official plugin issue tracker. No need of searching for a plugin’s tracker on the internet or copy-pasting long tracebacks to your browser.

bug report plugin

Installation

  • Install Report plugin from QGIS plugin offical repository
  • Create a GitHub account if you do not have any existing
  • Create a public access token GitHub with public_repo scope
  • Click on the Report plugin toolbar button, Configure link and copy generated GitHub access token to the configuration dialog.
  • Wait for next error and report the problem to the developers

Testing

  • Feel free to download DevNull plugin from QGIS plugin offical repository to be able to test Report plugin
  • After installation fo DevNull plugin, click on the new button /dev/null in your toolbar and watch how report plugin catches the exception
  • Report any number of testing issues to DevNull Issue tracker, they are going to be deleted anyway!

Notes

  • Report plugin works only with GitHub issue trackers
  • Plugins must have “tracker” metadata filled in, so the Report plugin can detect the plugin issue tracker correctly

When/if QGIS issue tracker moves to GitHub, a similar tool can be added to the core to automatically report bugs and crashes.

Setting Up and Configuring Discovery Plugin for QGIS

As a part of migrating to Open Source GIS, the Newcastle City Council has commissioned us to create a user friendly gazetteer plugin.

In this post, we will import the OS Open Names and configure the Discovery plugin to use the data.

In the following sections, we are going to set up a PostGIS database and import the OS Open Names, freely available from here.

To use the data with the Discovery plugin, we need to first set up a PostGIS database and load the data inside the geo-databases.

If you have an existing Postgresql/PostGIS server, you can skip the next section.

Postgresql/PostGIS installation and configuration

Installation

For MS Windows users, download and install Postgresql from here (32-bit) or here (64-bit) install the software.

During the installation, select the StackBuilder to install PostGIS, under Spatial Extensions.

If your StackBuilder fails to download PostGIS (in case your proxy server blocks it), you can download and install it manually.

Preparing the database

Now that installation is successful, create a new database as osdata. Make sure you add PostGIS extension to your database. You can do that by simply running the following command in the Query editor:

CREATE EXTENSION postgis;

Create osopennames as a new schema under osdata.

Preparing data

The OS Open Names comes in a zip file containing several CSV files.

Once the zip file extracted, there are 2 folders, one containing the header file (DOC) and the other containing the csv files (DATA).

To be able to import all the csv files in PostGIS, we can merge all the files including the header file.

You can move the header file (OS_Open_Names_Header.csv) from the DOC folder. To ensure, the file will appear first during the merge process you can rename it to 1_OS_Open_Names_Header.csv.

You can use Windows command prompt to merge the files. The following command merges all csv files to all_open_names.csv:

copy /b \*.csv all_open_names.csv

Loading data in PostGIS

There are several methods to import all_open_names.csv in PostGIS:

  1. Adding it as a delimited text layer in QGIS and then load it in PostGIS
  2. Importing it in PostGIS as a CSV and then using PostGIS' geometry to create points
  3. Using virtual layer and OGR2OGR library

In the example below, we explore the third option.

To create a virtual vector layer from your csv file, open a text editor, copy and paste the following lines and save it as all_open_names.vrt under DATA folder along with your csv file.

```

<OGRVRTDataSource>
    <OGRVRTLayer name="all_open_names">
        <SrcDataSource relativeToVRT="1">all_open_names.csv</SrcDataSource>
        <GeometryType>wkbPoint</GeometryType>
        <LayerSRS>EPSG:27700</LayerSRS>
        <GeometryField encoding="PointFromColumns" x="GEOMETRY_X" y="GEOMETRY_y"/>
    </OGRVRTLayer>
</OGRVRTDataSource>

```

In fact, you can use the virtual vector layer to merge all your csv files and skip the previous section!

You can then use ogr2ogr command from the OSGeo4W shell to import it to your PostGIS:

```

ogr2ogr -append  -a_srs EPSG:27700 -f "PostgreSQL" PG:"host=127.0.0.1 user=postgres dbname=osdata password=postgres active_schema=osopennames" -nln osnames all_open_names.vrt

```

Configuring Discovery plugin

First you need to install the plugin from the QGIS plugin repository.

Once the plugin installed, you should have a new toolbar. Click on discovery from the toolbar to open the configuration:

  1. For Connection, select OS Data
  2. For Shema, select opennames
  3. For Table, select osnames
  4. For Search Column, select Name1
  5. Select the option to Echo Search Column in Results
  6. For Display Columns select the followings:
    1. Name2
    2. DISTRICT_BOROUGH
    3. POSTCODE_DISTRICT
    4. COUNTRY
  7. For Geometry Column, select geom (or other columns depending on your QGIS or OGR versions)
  8. For BBOX Expresssion, type the following:

``` CASE

WHEN "MBR_XMIN" IS NOT NULL

THEN

MBR_XMIN || ',' || 
MBR_YMIN || ',' || 
MBR_XMAX || ',' || 
MBR_YMAX 

END ```

And you should be able to start using the gazetteer plugin after pressing OK.

For performance enhancement and other tips visit this page.

Crayfish 2.2: New Features

Crayfish 2.2 is out with lots of new features and new formats.

Here are the new features in more detail…

Plotting time series and cross sections

Profile tool plugin deemed not to be suitable for plotting time series. It lacks several features for visualising plots over s specific grid and generally to user interface is not very intuitive.

With the new plot options, user can plot time series at a point from the map or using an existing layer to generate plots. The tool is very flexible and allows user to export the layer to another format. In addition to time series, you can also generate cross section plots. Video below demonstrates the new feature in action:

As you can see, in the above video, more points can be added to the plot. To do so, you need to hold down CTRL key.

GRIB and NetCDF support

We have decided to expand the user base for Crayfish and allow meteorologists and oceanographers to view their temporal unstructured GRIB or NetCDF directly in Crayfish. Examples of those datasets can be found here and here

Visualising global temprature datasets (GRIB fromat) in Crayfish

Support for TELEMAC and HEC RAS 2D

We also added suppord for HEC RAS 2D and TELEMAC. Our initial benchmark shows that Crayfish is much faster than an other TELEMAC plugins we used. Plus, Crayfish is truly open source!

More vector, contour and mesh options

You can now filter vectors to display only values above/below a certain threshold.

Contour label and values in Crayfish are now similar to raster styling in QGIS. This will allow users to generate better legend directly from Crayfish for print or animation outputs.

For debugging, we have added option to allow users to label mesh and change symbology.

Mesh labels in Crayfish

Sponsor

We’d like to thank The Laboratory of Hydraulics, Hydrology and Glaciology (VAW) of ETH Zurich for sponsoring plotting feature.

Visualising Dam Breach in QGIS

Recently, I have come across the news about extent of damage and human loss, in case of the Mosul dam breach in Iraq.

Having all the tools, some freely available data and a free weekend, I have decided to carry out my own analysis.

Data

For the Digital Elevation Model (DEM), I used the data from JAXA Earth Observation Center (EORC).

The resolution of the DEM is approximately 30 metres. For more detailed analysis, a better dataset (e.g. LIDAR) would be ideal.

Methodology

I set up a 2-dimensional model with the following assumptions and parameters:

  • A square cell size (70 metres)
  • The dam was assumed to be full
  • Simulation time of 24 hours
  • The breach would occur after the first hour (1:00 hour in to the model run)
  • The dam would be completely breached after 1 hour (between 1:00 – 2:00 hour during the model run)

Results

Using Crayfish plugin for QGIS, the results can be animated to see the on-set of the flooding.

Note, that the first hour of the model run was used to initialise the water level within the dam. Hence, for the actual flood travel time, one hour should be deducted from the timing shown towards the bottom right corner of the video.

Below, you can see the flood extent generated based on my calculations and the extent shown here (red line in the main map) for Mosul.

Maximum flood extent due to the breach of the upstream dam in Mosul. (Click to enlarge)

Disclaimer: The calculation carried out for this demo was very approximate. Despite obtaining very similar results for Mosul compared to the original study, further checks and revisions are required.

Installing Third-party Python Modules in QGIS (Windows)

We’ve recently been developing a client plugin for forest management which relies on the Python SQLAlchemy module. When I installed the latest version of my colleague’s work I got a Python traceback error about a missing Python module (SQLAlchemy).

This blog is more of a note to myself for future reference of how to install third-party modules in QGIS' Python environment.

The Problem

Installing third-party modules in Windows is usually quite straight-forward, either download an installer (which will find the Python environment from the registry) or use easy_install from the Python setuptools.

The problem is that QGIS ships with its own Python installation that these methods cannot easily add to.

(Don’t get me wrong – I like the fact that each QGIS instance has its own Python environment as it keeps each version very self-contained).

The Method

So here’s the method I successfully used to install SQLAlchemy (which I’ll use here to install the lxml module):

  • Click Start
  • Type QGIS
  • Wait for the QGIS version / instance you want to modify to appear:

  • Right-click it, select Run as administrator

  • In QGIS, select Python Console from the Plugins menu
  • Download ez_setup.py from here.
  • Run the following code on the python console in QGIS:

``` from subprocess import call

Replace the path below to the location of ez_setup.py that

you just downloaded

call([‘python’, r'C:\Users\Pete\Downloads\ez_setup.py'])

This will install setuptools which is a package manager

The previous command should return 0 on success

Replace lxml with the package you wish to install

call([‘easy_install’, ‘lxml’])

Again this will return 0 on success

```

  • Restart QGIS (normally, not using the Run as administrator option)

The new module should now be available, we can test this (again on the Python console in QGIS):

import lxml

That’s it!

Trace Digitising in QGIS

Tracing in QGIS 2.14


Following the popularity of the AutoTrace plugin, we received a number of requests for additional features and to incorporate the tool into the main Advanced Digitising tools in QGIS.

Funding for the work was generously raised with the help of the community in 2015 and the feature landed in QGIS 2.14

Read more if you’d like to learn to use these new tools…

How to use Trace Digitising

Check out the Trace Digitising in QGIS for detailed instructions on how to use the tool.

Key features

The trace tool:

  • uses Dijkstra’s shortest path algorithm to find traceable routes
  • can trace routes over multiple distinct features
  • can be used with Advanced Digitising tools (e.g. reshaping)
  • can be enabled and disabled by pressing T on your keyboard while digitising
  • is fast and easy to use

The following video demonstrates how to use the tracing tool:

This video shows how you can make use of tracing while reshaping a polygon:

Sponsors

The project was kindly sponsored by the QGIS community. Special thanks to all those who contributed towards the development of this tool:

  • The Royal Borough of Windsor and Maidenhead
  • Neath Port Talbot County Borough Council
  • Ujaval Gandhi
  • Surrey Heath Borough Council
  • Matias Arnold
  • Northumberland National Park Authority
  • Buccleuch Estates Limited
  • Countryscape

Using Presets, Multiple Styles, Atlas With Overview Map

Our training courses are structured to give trainees step-by-step guide using the latest QGIS features. Here is an example of one our practicals for creating professional maps.

Trainees will learn how to

  • Assign multiple style to a layer
  • Working with presets
  • Working QGIS print composer
  • Generating maps using Atlas

Read on if you’d like to get a flavour of our training course practicals…

Creating professional maps with the Print Composer

QGIS' Print Composer allows users to create professional maps for printing. It supports legends, frames, logos and all the other features you’d expect to see in a print quality map.

In this example, we’ll use some of the most common features required to generate a map for printing.

We’ll first load the electricity usage data created in another tutorial, style it using a simple colour ramp, then create a new map composer complete with title and legend.

Click here to download the tutorial data. After downloading, ensure to extract the file.

Loading data in QGIS

The styling of a vector layer can be performed using a single style, or a varying style, based on the value of one or more attributes. We are going to first load the London energy usage layer prepared in an another tutorial by joining electricity usage with the LSOAs vector layer.

``` Start a new QGIS project

Add the MSOA_GL_ElecUsage.shp layer under extracted downloaded folder ```

The vector layer should now look like the figure below.

Energy consumption in London Energy consumption in London

Next, we are going to add a shapefile of London Boroughs to identify areas of high and low usage.

``` Add London_boroughs.shp from extracted downloaded folder

Open layer properties and under Style:

Set the style of the polygons to “No Brush” with a “Dash Line”. ```

Styles for London Boroughs vector layer (Click to enlarge)

Your map now should look like the figure below:

Energy consumption in London Energy consumption in London

In QGIS, you can apply multiple styles to a single layer. Using this method, we do not need to load the same layer multiple times. In this case, we need two styles for London boroughs layer:

  1. Without labels (as created above); to be used as an overview map

  2. With labels (next section); for the main map

We can save the current style (without label):

``` Right-click on London_boroughs layer from the layer tree and select Styles > Add

For the new style, type ‘no label’ and click OK ```

We can save the new styles (with labels) for the layer:

``` Right-click on London_boroughs layer from the layer tree and select Styles > Add

For the new style, type ‘with label’ and click OK ```

Changes in the next section (adding label) will be saved in with label style.

Labels

We’ll now add labels to each of the boroughs. The name of each borough is stored as the value of the NAME attribute.

``` Click on the London_boroughs layer in the ToC

Form the main menu, select Layer > Labelling ```

This window will allow us the set an attribute for label and also change font, size, rotation, buffer, etc.

Setting up labels on the London boroughs layer Setting up labels on the London boroughs layer

``` Enable the Label this layer with option

Set Field with labels to NAME

Under Text section, set the font to 7 pt

Under Buffer section:

  Enable Draw text buffer

  Set the Size to 0.5 mm

Click OK ```

You should now be able to see a label for each borough.

London boroughs with labels London boroughs with labels

Presets

In QGIS, you can create presets for layer visibility and styles. In this example we are going to create the following presets:

  • Map of LSOAs with the energy usage and borough boundaries

  • A separate preset for the London boroughs (without labels) which will be used as an “overview” for the print composer

To create the first preset:

``` Ensure both layers are visible

Right-click on London_boroughs and select Style > with label ```

On top of the legend tree in QGIS, you can see the icon for Manage Layer Visibility image

``` Click on the Manage Layer Visibility icon

From the drop-down menu, select Add preset…

For the Name of the new preset insert Energy map

Click OK ```

To ensure the preset has been correctly set, you can change the visibility of the layer, or change the style, and select the preset from the Manage Layer Visibility menu.

To create the second preset:

``` Hide All Layers from Manage Layer Visibility or simply press Ctrl+Shift+H

Turn on the visibility for London_boroughs

Right-click on London_boroughs and select Style > no label

On top of the legend tree in QGIS, click on Manage Layer Visibility

From the drop-down menu, select Add preset…

For the Name of the new preset insert overview map ```

London boroughs - without labels London boroughs

Now you should be able to switch between those two presets, and later use them as print composer frames.

Print Composer

Creating layout

The map is almost ready to be printed. Before printing, we’ll need to add a border, logos, legends, scale, etc.

There are quite a few steps involved in configuring the print composer so it’s recommended to regularly save your progress by saving the QGIS project.

``` From the main menu, select File > New Print Composer

A new window will appear for Composer title

Type Energy usage in London ```

The print composer title window

The print composer title window

The print composer will appear. See Figure.

PrintComposerWindowEmpty The print composer window

The print composer is split into two sections. Section 1 shows the map to be outputted and section 2 shows the settings for selected items in section 1.

The map we are going to produce is an A4 landscape PNG file.

In section 2, under Composition, for Presets, select A4(210 x 297 mm)

Now we need to add frames for the map extent and legend sections. You can add a rectangular shape by clicking on image from the main toolbar.

``` From the main toolbar, add a new rectangle

Click and drag a shape in section 1 ```

Now let’s change the position and size of the rectangle:

``` Ensure the rectangle shape is selected in section 1

Click on Item properties in section 2

Under Shape, select Position and size

Set the option as shown in Figure ```

Shape options

Shape options

``` Add second, third and fourth rectangles with the following information

Rectangle 2:

  X: 270

  Y: 145

  Width: 43.5

  Height: 60

  Reference Point: Centre

Rectangle 3:

  X: 270

  Y: 95

  Width: 43.5

  Height: 60

  Reference Point: Centre

Rectangle 4:

  X: 270

  Y: 105

  Width: 43.5

  Height: 200

  Reference Point: Centre

Lock the position of each window by right-clicking on each of them ```

Section 1 of the print composer should look like this:

The print composer with placeholders layout The print composer with placeholders layout

Adding image

Let’s add the logos.

``` From the main menu, select Layout > Add Image

Click within the middle rectangle on the right

An empty box will appear

In the right-hand panel (section 2), under the Item properties tab,

click on Main properties button 

For Path, click on … and browse to the folder where your images are located.

Select an image ```

You should now see something similar to the image below.

Logos added to the composer

Logos added to the composer

Adding text

Now we’ll add some text in the rectangle below the logos:

``` From the main menu, select Layout > Add Label

Click somewhere within the lower right-hand box
```

A new text box will appear, reading QGIS.

``` From the right-hand panel, click on Item properties

In Main properties, change the text to refer to the source of the data and copyright notices. ```

Copyright notices

Copyright notices

Adding map

Now we’ll add the map from the QGIS canvas.

``` From the main menu, select Layout > Add map

Drag a box in the main window ```

A map similar to the image below should now appear in the print composer.

To finalise the main map window, a few more changes will need to be made.

Adding map to the print composer Adding map to the print composer

``` From the right panel, click on the Item properties tab

  Under the Main properties, set the following values:

    Tick the box for Lock layers for map item see image below

  Under Extents:

    Tick the box for Controlled by atlas ```

Adding a preset for map view Adding a preset for map view

Add scalebar

To add a scalebar:

From the main menu, select Layout > Add Scalebar

Adding a scalebar to the map

Adding a scalebar to the map

A scalebar with default settings should appear on the map. See Figure [PrintComposerAddScalebar]. To change the scalebar:

``` Select the scalebar from the map

Click on Item Properties from the left-hand panel

Under Segments

   Set left 0 and right 2

   Set Size to 1000

   Set Height to 2 mm
```

Adding legend

Next, we’ll add a legend:

``` From the main menu, select Layout > Add Legend

Click somewhere within the upper right-hand box ```

QGIS will automatically add a legend for all the layers loaded in the canvas. We can edit the legend’s settings and remove the legend entries that we do not want to show. Manual edits of the legend text can also be performed to make it more readable.

To be able to edit the legend items, you first need to untick the box for Auto update. Use icons under the Legend items to edit or change the order of the items.

The default legend The default legend

``` From the right-hand panel, click on the Item properties

  Under Legend items:

    Select London_boroughs and then click on the red     minus sign

    Select MSOA_GL_ElecUsage and click on the edit     sign, a new window will appear.

    Change the content of Item text to Electricity

Usage 

    Click on the first value 2576.2400 – 3000.0000

and select edit sign to change the range to 
2000 – 3000   

    Repeat for the rest of the values to round them

and remove any unwanted zeros.   

```

Editing legend values

Editing legend values

Map with legend Map with legend

Add overview

To add an overview:

``` Add a new map (Map 1) to the place-holder between the map legend and the logos

For Map 1, from the Item properties,

Under Main properties

   Select Lock layers for map item

   From the visibility preset, select overview map

Under Overviews tab

   Click on the plus sign,

Overview 1 should be added to the overview list 

   Ensure Draw “Overview 1” overview is selected

   For Map frame, select Map 0 ```

Setting the preset view for the overview map Setting the preset view for the overview map

The map is now almost ready. You can export it as PNG or a PDF. Or you can keep reading for even more excitement!

The completed map The completed map

Atlas generation

The map is now ready to be printed or outputted as an image. But, we are going to use this layout as a template and auto-generate energy consumption for each borough.

To set the coverage (this is already set, but make sure the option is enabled):

``` While Map 0 (the main map) item is selected, click on Item properties from the right-hand panel

Select the option for Controlled by atlas generation ```

You can change the Margin around feature if you like, but we can keep at 10%. Further settings are also required:

``` Click on Atlas generation tab from the right-hand panel

Tick the box for Generate an atlas

Under Configuration, for Coverage layer select

London_boroughs 

Under Output, for Output filename expression type:

'energy_'|| "NAME"  

```

The expression for output will set the name of each output file. In this example we are going to have a name with ‘energy_’ combined with the value of NAME column (what do you think this value is?)

To generate the maps:

``` In the print composer, from the main menu, select Atlas > Export Atlas as Images…

Select a folder to output maps and press Choose ```

The process may take a while but you will eventually have 33 maps of the energy consumption for each borough. Note the overview for the maps and also the naming of the output files. See Figure beloq as an example of the maps created.

Final energy map for Kingston upon Thames generated by the Atlas Final energy map for Kingston upon Thames generated by the Atlas

Rule-based Labelling in QGIS 2.12

The new QGIS 2.12 (Lyon) will be out soon!

In this release, we have revamped the labelling engine and made it more flexible in-line with the rest of vector styling.

In this release, we have:

  • Revamped the labelling engine
  • Added support for mutually exclusive layer tree groups
  • Developed raster alignment tool

Read on for a how new rule-based labeling engine works…

For users

In previous versions of QGIS, users can select a field value or use an expression as labels for a vector layer. In QGIS 2.12, users can additionally define rules to label vector layers. Rule-based labelling works in the similar way as “Style”. A list of rules will be defined by users and they will be applied from top-to-bottom.

Field based labelling (Click to enlarge)

Rule based labelling (Click to enlarge)

To achieve the same effect in the earlier versions of QGIS, users should add a complex expression.The example below shows the expression used in earlier versions of QGIS:

CASE WHEN length( "htmlname" ) > 13 AND strpos("htmlname",' ') > 6 THEN replace("htmlname",' ',' ') WHEN length( "htmlname" ) > 20 AND "htmlname" LIKE '%Golf Course' THEN regexp_replace("htmlname",'Golf Course',' Golf Course') WHEN length( "htmlname" ) > 20 AND "htmlname" LIKE '%Nature Reserve' THEN regexp_replace("htmlname",'Nature Reserve',' Nature Reserve') WHEN length( "htmlname" ) > 20 AND "htmlname" LIKE '%Church Of England%' THEN regexp_replace("htmlname",'Church Of England',' Church Of England ') WHEN length( "htmlname" ) > 13 AND "htmlname" LIKE '% Of The %' THEN regexp_replace("htmlname",'Of The','Of The ') WHEN length( "htmlname" ) > 13 AND "htmlname" LIKE '% of %' AND "fontcolour" <> 2 AND "fontcolour" <> 4 THEN regexp_replace("htmlname",' of ',' of ') WHEN "htmlname" LIKE '%/%' THEN regexp_replace("htmlname",'/','/ ') WHEN length( "htmlname" ) > 30 THEN replace("htmlname",' ',' ') ELSE "htmlname" END

As you can see, without rule-based labelling users have to define each case and the labelling text. Additionally, with the cumbersome task of defining each case, users are limited to using specific labelling formats (e.g. font color, font size, visibility range, etc). With the new labelling engine, users can define their labels similarly to styles – in fact they can copy the rules from styling tabs and use it within the new labelling section!

Examples of label rules (Click to enlarge)

For developers

In order to facilitate addition of rule-based labelling, some internal changes were made to the QGIS labelling engine interface. The labelling is now driven by the new class QgsLabelingEngineV2 which may have several label providers associated with it. The label providers are objects derived from QgsAbstractLabelProvider and they are responsible for:

  1. providing “label features” that define properties of each label and geometry of the feature they represent; and

  2. drawing of labels at the positions determined by the engine.

Currently there are label provider implementations for diagrams, simple labelling and rule-based labelling.

The existing labelling engine (QgsPalLabeling class) is now built on top of the new labelling engine and works as a wrapper for it so that existing code that uses QgsPalLabeling still works.

As of now, the API for the new labelling engine is not considered as complete and therefore not available in Python. The idea is to make it easier to use, more polished and better prepared for the future use cases. It will be likely finished during the 2.14 release cycle and some changes may need to be postponed to QGIS 3.0 where backwards incompatible API changes will be allowed (as of now, QGIS 3.0 is being intensively discussed on QGIS developer mailing list).

Funding

This rule-based labeling has been funded by Tuscany Region (Italy). Special thanks also to the QGIS developers for their help with bug fixing.

Crayfish 2.1: New Features

New features keep being added to Crayfish. Now it is possible to export time variable grid as animation, add AnuGA results and visualise vectors on user-defined grids.

Here are the new features in more detail…

Export to animation

The ground works were done in Crayfish 2.0 for this feature. You can now generate animation from contours and vectors and export them as AVI. There are two methods of exporting to animation: basic and using QGIS print template (qpt).

With the basic option, you can define a title, a legend and a progress clock. Alternatively, for a smarter solution, you can set up a print composer with the benefit of all its rich features. The composer template (qpt) can be used as a frame layout for exporting your animation.

Below is an example of a multi-frame print composer template used to generate animation from a Crayfish layer.

AnuGA support

Crayfish 2.1 now supports SWW file format generated by AnuGA.

SWW files generated by AnuGA in Crayfish (Click to enlarge)

Vector on user-defined grid

With this option, users can define a grid and Crayfish will interpolate values and displays results on the custom grid. Images below show vectors on the outputted mesh and user-defined grid.

Vectors on default mesh in Crayfish (Click to enlarge)

Vectors on default mesh in Crayfish (Click to enlarge)

Vectors on a user-defined mesh in Crayfish (Click to enlarge)

Crayfish manual

With the ever-growing features in Crayfish, we have decided to dedicate a page on how to use Crayfish in QGIS. From the manual page, users can download a sample data and try the Crayfish features in QGIS.

Sponsors

We’d like to thank Maroondah City Council for sponsoring some of the great features in this release.

How to Use Function Editor in QGIS Field Calculator

In QGIS 2.8, there is a new option for users to add their own python function in the Field calculator. This is an extremely useful feature enabling users to populate data within the attribute table using customised python function.

Nathan wrote a blog post about the feature and how to write a python with arguments. But in QGIS 2.8, the function editor does not correctly support functions without arguments.

In the example below, we are going to calculate proportion of area for each SAC (Special Areas of Conservation) in Great Britain to the total area of the layer.

Add GB_SAC_20130918_Dissolved.shp layer to QGIS, right-click on the layer and open attribute table. Make the layer editable and click on the Field calculator. We are now going to create a new column (propArea) and populate proportionate area of each SAC to the total area of the layer.

Under Function Editor tab, click on New file and type area for the name and save the file. For the content, copy and paste the following lines:

"""
A custom function to calculate total area of the GIS layer.
This function has no arguments.
"""

from qgis.core import *
from qgis.gui import *
from qgis.utils import iface

@qgsfunction(args='auto', group='Custom')
def total_area(x, feature, parent):
    return sum( f.geometry().area() for f in iface.activeLayer().getFeatures() )

Click on Run Script to add total_area function to your Custom function list.

Now, click on Expression tab and type:

$area / total_area(0)

As you can see, we have passed 0 as an argument. If you click OK, your QGIS will freeze! As there are many features in the layer, the expression, calculates total area for each row.

Lets make the script a bit more elegant. Firstly, we need to add caching, so that area will be calculated only once and cached for the rest of operation. Secondly, we can make the script a bit more generic, so that we can use it to get the area of other loaded layers in QGIS:

from qgis.core import *
from qgis.gui import *
from qgis.utils import iface

cache = {}
def _layer_area(layer):
    """ Internal method """
    if layer.name() not in cache:
        area = sum( f.geometry().area() for f in layer.getFeatures() )
        cache[layer.name()] = area
    return cache[layer.name()]

@qgsfunction(args='auto', group='Custom')
def total_area(layer_name, feature, parent):
    for layer in iface.mapCanvas().layers():
        if layer.name() == layer_name:
            return _layer_area(layer)
    return 0

Now, click on Expression tab and type:

$area / total_area('GB_SAC_20130918_Dissolved')

This time, it should be quicker!

Crayfish 2.0: What's New!

Crayfish 2.0

After listening to user feedback we decided to do some major work on Crayfish. The changes include code refactoring, changes to the user interface, support for an additional file format, adding a vector and contour overlay, and a shiny new logo!

Read on for a look at some of the new features in more detail...

Time control

In the new version of Crayfish a time slider allows users to quickly browse through time. A drop-down menu allows the selection of an exact time.

Vector and contour overlay

In previous versions of Crayfish it was only possible to load contours and vectors from the same dataset. For example, it was not possible to show velocity vectors on top of depth contours. With the new Crayfish you can "unlock" the legend and choose different vectors or contours to be displayed. The video below demonstrates this in action.

Special times

Some datasets contain a special time-step outside the outputted time range. For example, Maximums and Minimums are stored at time 99999 in TUFLOW modelling package. Within the layer tree, additional time-steps items will now be shown if they exist within the dataset.

Additional file formats

We have added support for the XMDF file format. In addition, Hydro_AS 2D users should be able to open their files in the latest Crayfish.

New Python Module

We’ve refactored lots of code in the Crayfish library which makes it much easier to add support for further file formats and additional functionality. The Crayfish library now comes with a new Python module that allows easy manipulation with the mesh and results data – either in your custom scripts or within the QGIS Python console. For example, printing the coordinates of the nodes of a mesh together with their elevation takes just few lines of code:

import crayfish

m = crayfish.Mesh("/data/my_mesh.2dm")
o = m.dataset(0).output(0) # bed elevation data

for index, node in enumerate(m.nodes()):
print "Node XYZ: ", node.x, node.y, o.value(index)

New Profile tool plugin

If you use Profile tool plugin in QGIS, you can create a profile from the Crayfish layer and browse through the time. The profile gets updated as you change the output time.

Problems

If you have some feedback on our changes, suggestions for new functionality, or come across a bug, feel free to file a ticket on the issues page of the Crayfish github repository.

Sponsors

We’d like to thank Maroondah City Council for sponsoring some of the great features in this release.

Using Layer Visibility Presets in QGIS

One of the cool outcomes of our recent QGIS Legend Refactoring work was the ability to add "visibility presets". This feature allows users to save and load preset layer and group visibility settings.

Main canvas

First select the layers and groups, you’d like to manage visibility presets for. Now click at the top of the Layers panel and select Add preset … A new window will appear. Enter a name for the preset and click OK. Next time you click you should see the saved presets. The presets also save the visibility of categories in each layer.

In the video below, we create two presets. One for a specific site location and another zoomed to the outline of Great Britain which will be used as an overview in the print composer.

Print composer

The print composer now allows multiple map windows, each locked to a different layer visibility preset. This means a single map composer can show map windows with different layer arrangements.

In the video below, one preset is used for the main map and the other for an overview:

Crayfish 1.3: What's New!

Crayfish is becoming the must-have QGIS plugin for those working with the binary and ascii DAT/.2dm formats. Recently we had some requests from BASEMENT users to fully support the file format of their modelling package.

We have also been busy beta-testing our new illuvis flood risk communication service. Crayfish 1.3 integrates the illuvis client, allowing users to interact easily with this flood risk mapping service.

Read on for a look at some of the new features in more detail...

Advanced Styling

You can now turn off values below or above certain thresholds. This feature is very useful when you work with rainfall-runoff models and want to turn off values below e.g. 2 cm.

You can now set your colour ramps once, save them and apply them to other layers.

Export to Geo-TIFF

You can now easily export outputs from Crayfish to Geo-TIFF (.tif) files. Simply select the Quantity and Output time in the Crayfish viewer then right-click on the Crayfish layer in the QGIS Layers panel and select Export to grid…

Export to illuvis

For those users who have been using illuvis, you can now upload crayfish layers directly to the flood map publishing web service rather than first having to convert results to Geo-TIFF. If you would like to become an illuvis beta-tester, please contact us.

Support for BASEMENT

Users can now view outputs from the BASEMENT modelling package.

Binary package for Ubuntu

We love Linux and have finally managed to prepare crayfish binaries for Ubuntu 14.04. If you are using other distros, the compilation should be straight-forward using guide in the README file.

Problems

If you have some feedback or come across a bug, feel free to file a ticket on the issues page of the Crayfish github repository.

Sponsors

We’d like to thank The Laboratory of Hydraulics, Hydrology and Glaciology (VAW) of ETH Zurich for sponsoring some of the great features in this release.

Loading MasterMap: Free, Easy and Fast

We've developed a new graphical loading tool for OS MasterMap data focussing on usability and performance to make it easy to load national Ordnance Survey MasterMap datasets in a matter of hours.

OS Translator II - Load OS MasterMap into PostGIS quickly, easily and for free

The tool is OS Translator II - it makes use of the excellent GDAL library and is available now in the official QGIS Plugins repository.

This blog post talks about some simple benchmarks we've carried out.

Results

National load times were as follows:

  • MasterMap Topography (National) 20 hrs 21 mins 1
  • MasterMap ITN (National) < 6 mins

Installing PostgreSQL, PostGIS and QGIS took less than 10 minutes.

1 This is the most time-consuming test which filled the SSD on the first attempt. Importing to a tablespace on the main HDD completed after 20.3 hours but showed the import of tile 1592959-TR0585-5c3268.gz to have failed with this error. Until this issue is resolved the tile would need to be loaded and de-duplicated manually (e.g. using ogr2ogr to import and a SQL query to de-duplicate) to complete the dataset. De-duplication removes duplicate features caused by the chunking / supply process.

Comparison With Other Open Source Tools

We were curious as to how OS Translator II load times compared with other open loading methods so we did some basic tests using the "SU" tile of MasterMap Topography and ITN datasets and compared it with the popular Loader scripts. The results looked like this:

OS Translator II Benchmarks

Please note that OS Translator II had an unfair advantage in these tests as it automatically takes advantage of multiple-CPU cores whereas Loader presently does not.

Hardware and Software

We used the following hardware and software configuration:

  • CPU Intel Core i7 4790K (Haswell) @ 4GHz
  • Memory 32GB PC3-12800
  • Disk(s) Samsung 840 EVO 250GB SSD and Seagate Barracuda ST2000DM001 2TB HDD2
  • OS Microsoft Windows 7 Professional (64-bit)
  • PostgreSQL 9.4.1 (x64)
  • PostGIS 2.1.5 (x64)
  • QGIS 2.6.1 (Brighton)
  • OS Translator II 1.0
  • Python 2.7.9 (win32)
  • lxml 3.2.3 (win32)
  • Loader Master (067a511313, 20th February 2014)

2 Operating system and source gml.gz files located on the SSD and default PostgreSQL tablespace stored on secondary 2TB HDD.

PostgreSQL Configuration

The following changes were made to the default PostgreSQL configuration:

  • shared_buffers 512MB
  • work_mem 16MB
  • maintenance_work_mem 128MB / 1024MB3
  • checkpoint_segments 6
  • random_page_cost 2.0
  • fsync off

3 maintenance_work_mem was set to 1024MB for the national load of MasterMap Topography layer only.

Turning fsync off is dangerous and can lead to data loss in the event of an unexpected power outage. Always switch fsync back on after loading and never use this option on a database containing critical data.

  • Page 1 of 2 ( 34 posts )
  • >>

Back to Top

Sponsors