QGIS Planet

Mesh Data Abstraction Library (MDAL)

In the world of GIS, vector and raster are the most common data types to represent real world features. But not always, vectors and rasters can describe the complexity and nature of the data.

MDAL Mesh

Example of those datasets are:

  • Metrological data: e.g. temperature and wind speed
  • Hydrological data: flood depth and flow velocity
  • Oceanography: salinity and wave flux

MDAL arrows and contours

In the past, we introduced Crayfish plugin to handle unstructured grids, usually with temporal and other components. The main issues with Crayfish plugin were:

  • Crayfish plugin had its own renderer. Therefore, we had to create binaries and ship them with the plugin. This was not ideal, as not all the platforms (e.g. MacOS) was not supported.
  • Crayfish plugin loaded all the data in the memory. For small files, this was not an issue. But there was not a solution for larger files.

Why MDAL?

For more details of the rational behind introducing MDAL, see the QGIS Enhancement Proposal with great suggestions and feedback from the developers.

TL;DR: we needed a new abstraction library to handle the mesh data formats similar to GDAL/OGR dealing with rasters and vectors.

Current state

After the feedback from the QGIS community, we have introduced MDAL library earlier this year. It took us a bit of time to put the infrastructure in place. There are already some formats available from MDAL. The library was integrated to QGIS from 3.2. But extensive improvements and new features were added during 3.4.

There are still more formats to support. In QGIS, we will also need to enhance the spatial indexing and performance of the driver in general. For more information visit MDAL website.

How to work with MDAL layer in QGIS

Here is a mesh layer you can use to load in QGIS. For more data, you can visit the ECMWF(or Copernicus) or NASA websites.

In QGIS, from the main menu > Layer > Data Source Manager. A new window will appear. From the left panel, click on Mesh layer Mesh and point to the downloaded file.

You should be able to see a figure similar to the one below:

Mesh layer in QGIS

To view quantities within the mesh, you can open the Layer Styling Panel. Click on symbology Styling tab:

  • First select symbology and deactivate Native Mesh Rendering
  • Click on general and under groups, towards the right of 10 metre wind click on contour and vector to view the quantities.
  • Below the Groups, you can move the slider to browse through time of the dateset(s).
  • To change style of the contours and vectors you can click on contour and vector tabs and adjust the color ramps and vector sizes accordingly.

Mesh layer in QGIS

What happens to Crayfish plugin?

We have recently released a new version of Crayfish plugin to work directly with mesh layer in QGIS. Crayfish is now based on Python only and can be installed on all main platforms (including MacOS).

Future developments

We are planning to add mesh calculator (either as a core QGIS feature or Crayfish/Processing plugin). There will be more formats in the pipeline to support in MDAL too.

If you’d like to add support for your mesh layer to MDAL, you can follow the examples on the github repository. We are always happy to help.

New 3D features in QGIS 3.4

Thanks to the great response from the QGIS community to our crowdfunding call, we have added several new features to the upcoming release (3.4).

Here are the highlights of the features:

Print layout

With this new feature, you can embed a 3D scene to the print layout. In addition, there are camera and view setting tools available to adjust the scene from within the map frame. This will allow you to generate high resolution outputs similar to the other map frames.

QGIS 3D configuration

Animation

You can create an animation based on a set of keyframes - camera positions at particular times. QGIS 3D then interpolates the camera positions/rotations in between the keyframes.

To create the keyframes, first set the scene for your map, by rotating, zooming or moving the camera. Then assign a time to the frame. There are several methods for interpolations between keyframes.

QGIS 3D configuration

Identify tool in 3d map

A new Identify Tool was introduced to 3D map. Using this tool, you can inspect features from the 3D scene. Thanks to our friends from Faunalia for funding this feature.

3D lines

If you have a 3D linestring, you can now use the elevation of vertices to display it in 3D map.

QGIS 3D configuration

Camera control

There are more control on how you can pan, rotate and zoom the camera within the scene:

  • Page up/down: to move the camera up/down: Page up/down keys for moving the camera
  • Free camera movements: also possible to look from below: Page up/down keys for moving the camera
  • Shift key and drag: the camera rotates around a point on terrain: Page up/down keys for moving the camera
  • Ctrl key and drag: this will rotate camera while it stays in one position Page up/down keys for moving the camera

In addition, the point towards which the camera is looking is dynamically adjusted based on the terrain. The rotation in scenes where a DEM used for terrain has been enhanced significantly. This is more pronounced with presence of higher hills in the terrain.

Further development

There are still some exciting features from the crowdfunding which will be incorporated in QGIS 3.6. You can check the QGIS development or follow us on Twitter.

Working with QGIS 3D - Part 1

In QGIS 3, we have introduced support for 3D canvas. Most of the functionalities are intuitive and easy to use. But there are some configuration options which are hidden and require a bit of more in-depth explanation for users and developers.

This blog post and the follow-up ones will discuss a range of topics: data sources, 3D canvas navigation, configuration, working with various types of layers, styling and more!

Data Sources

To work with QGIS 3D, you need data: rasters and vectors. We will use digital terrain model rasters for 3D visualisation purpose. You can download SRTM data from the SRTM Tile Downloader.

For vectors, you can use any point, line and polygon data. There are different methods of creating 3D objects from each data type in the 3D canvas. If you want true 3D data representing buildings, you can download CityGML data from the list of open CityGML datasets.

For the purpose of these blog posts, we will use SRTM data for Mont Blanc and CityGML data for Berlin.

Note: You need to use a projected coordinate reference system in metres (or feet in case you belong to one of these countries) for your data and canvas to be able to use QGIS 3D.

3D Canvas and Navigation

To start with, we are going to add the terrain model for Mont Blanc to the QGIS canvas. Bing Aerial photo (as XYZ tiles layer) was also loaded in QGIS.

Mont Blanc terrain model with Bing aerial in QGIS.
(Click to enlarge)

To view the 3D canvas, in the main menu select View > New 3D Map View

A floating QGIS panel will appear. You can drag the panel to the bottom part of your canvas to dock it.

3D and 2D canvases in QGIS 3.
(Click to enlarge)

To start with, the 3D view shows the same extent and view as seen in the 2D canvas. Also note that there is no dedicated toolbar for navigation in the 3D canvas. You can zoom in/out and pan in the 3D canvas in the same way as in the main 2D canvas:

  • Move around map
    • by dragging the map with left mouse button pressed
    • by using up/down/left/right keys
  • Zoom map in/out
    • by using the mouse wheel
    • by dragging mouse up/down with right mouse button pressed

The following additional options allow you to explore the map in 3D:

  • Tilt / rotate camera
    • by dragging the mouse with middle mouse button pressed
    • by pressing Shift and dragging the mouse with left mouse button pressed
    • by pressing Shift and using up/down/left/right keys

To reset the camera view, click Zoom Full button in the 3D canvas panel.

Terrain Configuration

You can use a terrain raster to represent 3D elevation in your canvas. It is expected that such raster layer contains one band where each raster cell represents elevation. To do that, click Options button to open a new window with 3D view configuration. After selecting your raster layer for Elevation and clicking OK, you should be able to see Mont Blanc in the 3D view:

3D view of Mont Blanc.
(Click to enlarge)

Advanced Configuration

In the configuration window there are various other options to fine-tune the 3D scene - let’s have a closer look at their meaning. Before diving into the details, it is worth noting that terrain in 3D view is represented by a hierarchy of terrain tiles and as the camera moves closer to the terrain, existing tiles that do not have sufficient detail are replaced by smaller tiles with more details. Each tile has mesh geometry derived from the elevation raster layer and texture created by rendering 2D map for the extent of the tile.

QGIS 3D configuration

Here is the complete list of the configuration options and their meaning:

  • Elevation: Raster to be used for generation of terrain.
  • Vertical scale: Scale factor for vertical axis. Raising the scale will make even small hills look like mountains!
  • Tile resolution: How many samples from the terrain raster layer to use for each tile. The value of 16 px means that geometry of each tile will be built from 16x16 elevation samples. Higher number creates more detailed terrain tiles at the expense of increased rendering complexity.
  • Skirt height: Sometimes it is possible to see small cracks between tiles of the terrain. Raising this value will add vertical walls (“skirts”) around terrain tiles to hide the cracks.
  • Map tile resolution: Width and height of 2D map images used as textures for terrain tiles. The value of 256 px means that each tile will have map rendered into image of 256x256 pixels. Higher number creates more detailed terrain tiles at the expense of increased rendering complexity.
  • Max. screen error: Determines threshold when existing terrain tiles are swapped to more detailed ones (and vice versa) - i.e. how soon 3D view will use higher quality tiles. Lower number means more details in the scene at the expense of increased rendering complexity.
  • Max. ground error: Tells the 3D view at what resolution of terrain tiles it is fine to stop dividing them into more detailed tiles (because splitting them would not introduce any extra detail anyway). This value limits the depth of the hierarchy of tiles: lower value makes the hierarchy depth, increasing rendering complexity.
  • Zoom levels: Show how many zoom levels will be used (depends on map tile resolution and max. ground error).
  • Show labels: Toggles the map labels on/off
  • Show map tile info: Adds border and tile numbers to terrain tiles (useful for troubleshooting terrain issues)
  • Show bounding boxes: Shows 3D bounding boxes of terrain tiles (useful for troubleshooting terrain issues)

Crowdfunding: More QGIS 3D

We are excited to launch a new crowdfunding campaign to bring lots of new features to QGIS 3D!

QGIS 3D

Here is a brief summary of what to expect if the campaign will be successful:

  • Print layout support
  • Camera animation support
  • Better camera control
  • Rule-based 3D rendering
  • Earth as a globe
  • Global terrain coverage
  • Skybox
  • Map themes for terrain
  • Configuration of lights
  • Loading map tiles in parallel

The target amount is 12,500 € and the campaign will be active until 16 March 2018.

Please have a look at the dedicated page More QGIS 3D for further details and help us spread the word!

Working with climate data in QGIS

In this tutorial, we are going to explore methods of visualising climate data within QGIS using Crayfish plugin. We are going to use September 2017 datasets from NASA which will include Hurricane Irma.

Downloading data

Note: You can download the processed data here. and skip to Viewing data in QGIS.

There are several resources available to obtain climate data. Due to the large volume of data and number of variables, the providers usually offer APIs to interact with dataset repository. But, you can download some datasets directly from their websites.

ECMWF and NASA data portals offering such services.

In this tutorial, we are using NASA portal to download NLDAS Secondary Forcing data for September 2017. Follow the instructions to batch download all the GRIB files using the script provided on their website.

Manipulating data

Each grb file downloaded in the previous step is only for one time step containing multiple variables. You can extract certain variable and also merge the files using Climate Data Operators:

cdo mergetime *.grb september.grb

cdo select,name=var61,var33,var34,var11 september.grb september_prec_wind_temp.grb

The first command will merge all the times and generates a single GRIB file containing all the time step. The second command, extracts temperature, precipitation and wind data.

Viewing data in QGIS

To view the data, you will need Crayfish plugin. Download and install the plugin from the QGIS plugin repository.

Add the GRIB file as a Crayfish layer (under Plugins > Crayfish > Add Crayfish Layer). You should be able to see precipitation, temperature and wind data from the Crayfish panel. Note that the wind data contains vector in addition to grid.

You can use the slider to move the time. Read more on how to use Crayfish plugin.

To view wind data as trace animation, move the slider time to 12.09.2017 07:00 (or 319:00:16.04 if your time is set to relative) and select Vector Options. Set the parameters according to the dialog window below:

Vector Options

You should now be able to see live traces of the wind from your QGIS canvas:

GIF animation of Irma

You can plot the time series by clicking on the plot tool from Crayfish panel:

To generate an animation of precipitation and wind, you need to first click on from Crayfish panel and then enable wind vectors and precipitation grid. From Plugins > Crayfish > Export to animation you can generate animation of your time series.

Support funding Crayfish port to QGIS 3

If you are interested in using Crayfish in QGIS 3, help with this crowd-funding.

Working With Climate Data in QGIS

In this tutorial, we are going to explore methods of visualising climate data within QGIS using Crayfish plugin. We are going to use September 2017 datasets from NASA which will include Hurricane Irma.

Downloading data

Note: You can download the processed data here. and skip to Viewing data in QGIS.

There are several resources available to obtain climate data. Due to the large volume of data and number of variables, the providers usually offer APIs to interact with dataset repository. But, you can download some datasets directly from their websites.

ECMWF and NASA data portals offering such services.

In this tutorial, we are using NASA portal to download NLDAS Secondary Forcing data for September 2017. Follow the instructions to batch download all the GRIB files using the script provided on their website.

Manipulating data

Each grb file downloaded in the previous step is only for one time step containing multiple variables. You can extract certain variable and also merge the files using Climate Data Operators:

cdo mergetime *.grb september.grb

cdo select,name=var61,var33,var34,var11 september.grb september_prec_wind_temp.grb

The first command will merge all the times and generates a single GRIB file containing all the time step. The second command, extracts temperature, precipitation and wind data.

Viewing data in QGIS

To view the data, you will need Crayfish plugin. Download and install the plugin from the QGIS plugin repository.

Add the GRIB file as a Crayfish layer (under Plugins > Crayfish > Add Crayfish Layer). You should be able to see precipitation, temperature and wind data from the Crayfish panel. Note that the wind data contains vector in addition to grid.

You can use the slider to move the time. Read more on how to use Crayfish plugin.

To view wind data as trace animation, move the slider time to 12.09.2017 07:00 (or 319:00:16.04 if your time is set to relative) and select Vector Options. Set the parameters according to the dialog window below:

Vector Options

You should now be able to see live traces of the wind from your QGIS canvas:

GIF animation of Irma

You can plot the time series by clicking on the plot tool from Crayfish panel:

To generate an animation of precipitation and wind, you need to first click on from Crayfish panel and then enable wind vectors and precipitation grid. From Plugins > Crayfish > Export to animation you can generate animation of your time series.

Support funding Crayfish port to QGIS 3

If you are interested in using Crayfish in QGIS 3, help with this crowd-funding.

Crayfish New Release and Future Plan

Crayfish 2.7 is out with grid calculator, new formats and several enhancements. This will likely to be the last version of Crayfish for QGIS 2.x. We have started a crowd-funding campaign to port Crayfish to QGIS 3.

Crayfish for QGIS 3.2

Crayfish has grown over the past few years and now handles several file formats. There are various functions within the plugin to analyse and process the data too. Instead of porting the plugin directly to QGIS 3, we decided to refactor the code with the possibility of incorporating the renderer directly in QGIS core. We have started a crowd-funding campaign and hoping the software vendors and organisations who are benefiting from the plugin will fund the effort:

http://www.lutraconsulting.co.uk/crowdfunding/qgis-crayfish-3/

Grid calculator

We have implemented a new grid calculator for Crayfish layer. Users can now perform various operations on one or multiple layers. The output can be filtered by time or spatially. In addition, we have added logical and aggregating operators. The result is currently saved as a binary XMS file. Read Crayfish wiki page for more details.

Grid calculator

Enhancements

Here is a list of enhancements for this release: – Better handling netcdf time – Resolving rendering issues with some file formats – Better support for Hydro_AS-2D files – Smoother area contour export

Sponsors

This release was funded by Austrian Ministry of Agriculture, Forestry, Environment and Water Management

Feedback and bug report

If you have any problem with Crayfish, please do not email us directly and consider filing a bug here: https://github.com/lutraconsulting/qgis-crayfish-plugin/issues

Crayfish new release and future plan

Crayfish 2.7 is out with grid calculator, new formats and several enhancements. This will likely to be the last version of Crayfish for QGIS 2.x. We have started a crowd-funding campaign to port Crayfish to QGIS 3.

Crayfish for QGIS 3.2

Crayfish has grown over the past few years and now handles several file formats. There are various functions within the plugin to analyse and process the data too. Instead of porting the plugin directly to QGIS 3, we decided to refactor the code with the possibility of incorporating the renderer directly in QGIS core. We have started a crowd-funding campaign and hoping the software vendors and organisations who are benefiting from the plugin will fund the effort:

https://www.lutraconsulting.co.uk/crowdfunding/qgis-crayfish-3/

Grid calculator

We have implemented a new grid calculator for Crayfish layer. Users can now perform various operations on one or multiple layers. The output can be filtered by time or spatially. In addition, we have added logical and aggregating operators. The result is currently saved as a binary XMS file. Read Crayfish wiki page for more details.

Grid calculator

Enhancements

Here is a list of enhancements for this release:

  • Better handling netcdf time
  • Resolving rendering issues with some file formats
  • Better support for Hydro_AS-2D files
  • Smoother area contour export

Sponsors

This release was funded by Austrian Ministry of Agriculture, Forestry, Environment and Water Management

Feedback and bug report

If you have any problem with Crayfish, please do not email us directly and consider filing a bug here: https://github.com/lutraconsulting/qgis-crayfish-plugin/issues

How to view buildings in QGIS3D

With support for QGIS3D canvas, you can represent your vectors in a number of ways. In this post, we will walk you through how to render vectors as 3D objects.

Data

You can use CityGML or ESRI Multipatch, where the height of buildings are stored within the feature. For the purpose of this example, we are going to view New York buildings using ESRI Multipatch format. You can download the data from here:

http://maps.nyc.gov/download/3dmodel/DA_WISE_Multipatch.zip

After unziping the file, you will have several gdb files. You can use GDAL/OGR to convert gdb to Geopackage (or ESRI Shapefile). We should also convert the geometry type from multipatch to multipolygon. In Microsoft Windows gdal/ogr commands are available from the OSGEO4W command line.

ogr2ogr -f GPKG ny_buildings3d.gpkg DA1_3D_Buildings_Multipatch.gdb -nlt multipolygon

You can append the remaining gdb files to the existing Geopackage:

ogr2ogr -append -f GPKG ny_buildings3d.gpkg DA2_3D_Buildings_Multipatch.gdb -nlt multipolygon

Alternatively, you can write a simple batch script to loop through the files.

Note: we used gdal/ogr from trunk (2.3.0dev).

If you know of any other data sources, please send us an email so we can compile a full list of supported formats.

Viewing data

To be able to use QGIS3D, you need to install the latest version of QGIS using OSGEO4W or other installer for your platform.

Add the Geopackage containing the buildings to your QGIS. In QGIS, from the main menu, click on **View > New 3D Map View **

A new view, similar to your 2D canvas will be added to the bottom of your canvas. To be able to extrude the buildings, we need to enable 3D styling of the building layer.

Ensure your Style panel is enabled (this is usually located on the right hand side of the canvas). Select 3D View tab and tick the box for Enable 3D renderer for building layer.

Vector 3D styling

To navigate in 3D canvas, you can use Shift key + the wheel button on your mouse device.

3D view

How to View Buildings in QGIS3D

With support for QGIS3D canvas, you can represent your vectors in a number of ways. In this post, we will walk you through how to render vectors as 3D objects.

Data

You can use CityGML or ESRI Multipatch, where the height of buildings are stored within the feature. For the purpose of this example, we are going to view New York buildings using ESRI Multipatch format. You can download the data from here:

http://maps.nyc.gov/download/3dmodel/DA_WISE_Multipatch.zip

After unziping the file, you will have several gdb files. You can use GDAL/OGR to convert gdb to Geopackage (or ESRI Shapefile). We should also convert the geometry type from multipatch to multipolygon. In Microsoft Windows gdal/ogr commands are available from the OSGEO4W command line.

ogr2ogr -f GPKG ny_buildings3d.gpkg DA1_3D_Buildings_Multipatch.gdb -nlt multipolygon You can append the remaining gdb files to the existing Geopackage:

ogr2ogr -append -f GPKG ny_buildings3d.gpkg DA2_3D_Buildings_Multipatch.gdb -nlt multipolygon

Alternatively, you can write a simple batch script to loop through the files.

Note: we used gdal/ogr from trunk (2.3.0dev).

If you know of any other data sources, please send us an email so we can compile a full list of supported formats.

Viewing data

To be able to use QGIS3D, you need to install the latest version of QGIS using OSGEO4W or other installer for your platform.

Add the Geopackage containing the buildings to your QGIS. In QGIS, from the main menu, click on View > New 3D Map View

A new view, similar to your 2D canvas will be added to the bottom of your canvas. To be able to extrude the buildings, we need to enable 3D styling of the building layer.

Ensure your Style panel is enabled (this is usually located on the right hand side of the canvas). Select 3D View tab and tick the box for Enable 3D renderer for building layer.

Vector 3D styling

To navigate in 3D canvas, you can use Shift key + the wheel button on your mouse device.

3D view

Building QGIS3D on (K)ubuntu 16.04

QGIS support for 3D canvas is ready for testing. A possible hurdle in getting QGIS compiled with 3D support may be the fact that we require Qt in version at least 5.8 and it is recommended to use Qt 5.9 which introduces further enhancements. The current QGIS master branch (to be 3.0 release) is usually built against earlier versions of Qt. For example in Ubuntu 16.04, the default Qt package version is 5.5.

Continue reading for more detail on how to build QGIS with the latest Qt on Ubuntu …

Build of QGIS

The default Qt (from official repositories) on (K)Ubuntu 16.04 is too old and does not include the new Qt 3D framework. We build QGIS with Qt 5.9.1. We are going to install QT to /opt/Qt5.9.1/ and QGIS dependencies built with Qt5.9 to /opt/qt59_libs, so make sure you have these folders created and ready to use.

Qt 5.9.1

To add Qt 5.9.1, we can use a ppa:

sudo add-apt-repository ppa:beineri/opt-qt591-xenial sudo apt-get update sudo apt-get install qt59-meta-full This will install Qt 5.9.1 side-by-side your current system Qt under /opt folder. You can later remove the package without affecting dependencies in your system.

alternatively you can download QT 5.9.1 installer from http://download.qt.io/official_releases/qt/5.9/5.9.1/qt-opensource-linux-x64-5.9.1.run and install it to the same location.

Qwt 6.1.3

Another dependency is Qwt. You can download the package and build it with Qt 5.9.1. To download the package, click here: https://sourceforge.net/projects/qwt/files/qwt/6.1.3/ Make a new folder and move the zip file there:

mkdir /tmp/qgis_deps mv ~/Downloads/qwt-6.1.3.zip /tmp/qgis_deps cd /tmp/qgis_deps unzip qwt-6.1.3.zip cd qwt-6.1.3

We need to define the prefix path. To do that, open qwtconfig.pri in a text editor and change the prefix path:

nano qwtconfig.pri

change QWT_INSTALL_PREFIX = /opt/qt59_libs/qwt-6.1.3 (more occurrences in the file!)

You can now compile the project:

/opt/qt59/bin/qmake qwt.pro make -j4 make install

Check if the library has been installed correctly: ls /opt/qt59_libs/qwt-6.1.3

QScintilla2 2.10.1

Use the compressed file from here: https://www.riverbankcomputing.com/software/qscintilla/download

Download and copy to /tmp/qgis_deps mv ~/Downloads/QScintilla_gpl-2.10.1.tar.gz /tmp/qgis_deps/ cd /tmp/qgis_deps tar xvzf QScintilla_gpl-2.10.1.tar.gz cd QScintilla_gpl-2.10.1/Qt4Qt5/ /opt/qt59/bin/qmake qscintilla.pro make -j4 sudo make install

You should now have the compiled qscintilla in the following path:

ls /opt/qt59/lib/libqscintilla2_qt5.so

QGIS

First clone (or add as another remote) QGIS fork wonder-sk/QGIS and change branch to 3d

git clone [email protected]:wonder-sk/QGIS.git cd QGIS git checkout 3d

Now you can follow standard instructions on QGIS repo for building the applications: https://raw.githubusercontent.com/qgis/QGIS/master/INSTALL

Once you have created the build directory (after step https://github.com/qgis/QGIS/blob/master/INSTALL#L266) you need to configure the cmake with the following options: CMAKE_PREFIX_PATH=/opt/qt59/lib/cmake cmake \ -GNinja \ -DCMAKE_BUILD_TYPE=Debug \ -DCMAKE_INSTALL_PREFIX=${HOME}/apps \ -DWITH_3D=TRUE \ -DWITH_QTWEBKIT=FALSE \ -DENABLE_TESTS=FALSE \ -DWITH_QWTPOLAR=FALSE \ -DWITH_BINDINGS=FALSE \ -DQWT_LIBRARY=/opt/qt59_libs/qwt-6.1.3/lib/libqwt.so \ -DQWT_INCLUDE_DIR=/opt/qt59_libs/qwt-6.1.3/include \ -DQSCINTILLA_LIBRARY=/opt/qt59/lib/libqscintilla2_qt5.so \ -QSCINTILLA_INCLUDE_DIR=/opt/qt59/include \ ..

The new flag is WITH_3D=TRUE.

In the output, make sure it has found built libraries (NOT Qt 5.7 line) -- Found Qt version: 5.9 -- Found Qwt: /opt/qt59_libs/qwt-6.1.3/lib/libqwt.so (6.1.3) -- Found QScintilla2: /opt/Qt5.9.1/5.9.1/gcc_64/lib/libqscintilla2_qt5.so (2.10.1)

Note that if you are using your own compiled version of GDAL, you need to define it using this flag: -DGDAL_CONFIG=/PATH/TO/bin/gdal-config

If all dependencies are detected properly, you should be able to build QGIS using ninja:

ninja

To run QGIS from your build folder:

cd output/bin ./qgis

To verify that you are using the right version of QGIS, you can go to Help > About and check which version of Qt your application has been built against.

Loading the data

Now in QGIS, open 3D Canvas in menu: View->New 3D Map View. For 3D styling of vector layers, open Layer Styling dock widget and enable 3D Renderer in the newly added tab with 3D cube icon.

Building QGIS3D on (K)ubuntu 16.04

QGIS support for 3D canvas is ready for testing. A possible hurdle in getting QGIS compiled with 3D support may be the fact that we require Qt in version at least 5.8 and it is recommended to use Qt 5.9 which introduces further enhancements. The current QGIS master branch (to be 3.0 release) is usually built against earlier versions of Qt. For example in Ubuntu 16.04, the default Qt package version is 5.5.

Continue reading for more detail on how to build QGIS with the latest Qt on Ubuntu …

Build of QGIS

The default Qt (from official repositories) on (K)Ubuntu 16.04 is too old and does not include the new Qt 3D framework. We build QGIS with Qt 5.9.1. We are going to install QT to /opt/Qt5.9.1/ and QGIS dependencies built with Qt5.9 to /opt/qt59_libs, so make sure you have these folders created and ready to use.

Qt 5.9.1

To add Qt 5.9.1, we can use a ppa:

sudo add-apt-repository ppa:beineri/opt-qt591-xenial
sudo apt-get update
sudo apt-get install qt59-meta-full

This will install Qt 5.9.1 side-by-side your current system Qt under /opt folder. You can later remove the package without affecting dependencies in your system.

alternatively you can download QT 5.9.1 installer from http://download.qt.io/official_releases/qt/5.9/5.9.1/qt-opensource-linux-x64-5.9.1.run and install it to the same location.

Qwt 6.1.3

Another dependency is Qwt. You can download the package and build it with Qt 5.9.1. To download the package, click here: https://sourceforge.net/projects/qwt/files/qwt/6.1.3/ Make a new folder and move the zip file there:

mkdir /tmp/qgis_deps
mv ~/Downloads/qwt-6.1.3.zip /tmp/qgis_deps
cd /tmp/qgis_deps
unzip qwt-6.1.3.zip
cd qwt-6.1.3

We need to define the prefix path. To do that, open qwtconfig.pri in a text editor and change the prefix path:

nano qwtconfig.pri

change QWT_INSTALL_PREFIX = /opt/qt59_libs/qwt-6.1.3 (more occurrences in the file!)

You can now compile the project:

/opt/qt59/bin/qmake qwt.pro
make -j4
make install

Check if the library has been installed correctly:

ls /opt/qt59_libs/qwt-6.1.3

QScintilla2 2.10.1

Use the compressed file from here: https://www.riverbankcomputing.com/software/qscintilla/download

Download and copy to /tmp/qgis_deps

mv ~/Downloads/QScintilla_gpl-2.10.1.tar.gz /tmp/qgis_deps/
cd /tmp/qgis_deps
tar xvzf QScintilla_gpl-2.10.1.tar.gz
cd QScintilla_gpl-2.10.1/Qt4Qt5/
/opt/qt59/bin/qmake qscintilla.pro
make -j4
sudo make install

You should now have the compiled qscintilla in the following path:

ls /opt/qt59/lib/libqscintilla2_qt5.so

QGIS

First clone (or add as another remote) QGIS fork wonder-sk/QGIS and change branch to 3d

git clone [email protected]:wonder-sk/QGIS.git
cd QGIS
git checkout 3d

Now you can follow standard instructions on QGIS repo for building the applications: https://raw.githubusercontent.com/qgis/QGIS/master/INSTALL

Once you have created the build directory (after step https://github.com/qgis/QGIS/blob/master/INSTALL#L266) you need to configure the cmake with the following options:

CMAKE_PREFIX_PATH=/opt/qt59/lib/cmake cmake \
   -GNinja \
   -DCMAKE_BUILD_TYPE=Debug \
   -DCMAKE_INSTALL_PREFIX=${HOME}/apps \
   -DWITH_3D=TRUE \
   -DWITH_QTWEBKIT=FALSE \
   -DENABLE_TESTS=FALSE \
   -DWITH_QWTPOLAR=FALSE \
   -DWITH_BINDINGS=FALSE \
   -DQWT_LIBRARY=/opt/qt59_libs/qwt-6.1.3/lib/libqwt.so \
   -DQWT_INCLUDE_DIR=/opt/qt59_libs/qwt-6.1.3/include \
   -DQSCINTILLA_LIBRARY=/opt/qt59/lib/libqscintilla2_qt5.so \
   -QSCINTILLA_INCLUDE_DIR=/opt/qt59/include \
   ..

The new flag is WITH_3D=TRUE.

In the output, make sure it has found built libraries (NOT Qt 5.7 line)

-- Found Qt version: 5.9
-- Found Qwt: /opt/qt59_libs/qwt-6.1.3/lib/libqwt.so (6.1.3)
-- Found QScintilla2: /opt/Qt5.9.1/5.9.1/gcc_64/lib/libqscintilla2_qt5.so (2.10.1)

Note that if you are using your own compiled version of GDAL, you need to define it using this flag: -DGDAL_CONFIG=/PATH/TO/bin/gdal-config

If all dependencies are detected properly, you should be able to build QGIS using ninja:

ninja

To run QGIS from your build folder:

cd output/bin
./qgis

To verify that you are using the right version of QGIS, you can go to Help > About and check which version of Qt your application has been built against.

Loading the data

Now in QGIS, open 3D Canvas in menu: View->New 3D Map View. For 3D styling of vector layers, open Layer Styling dock widget and enable 3D Renderer in the newly added tab with 3D cube icon.

Crayfish 2.5: New features

A new release of Crayfish is now available from QGIS plugin repo.

Here are the new features in more detail…

Trace animation

We have added support for live trace animation on QGIS canvas:

Please note that export to trace animation (as an avi file) is not supported yet. To change vector style to traces, please refer to the Crayfish manual.

Support for UGRID

UGRID format for Deltares modelling packages are now supported in Crayfish. In addition to 2D mesh, UGRID also supports 1D mesh. Support for 1D mesh is experimental in this release.

FLO-2D HDF format

In addition to ASCII FLO-2D files, Crayfish can handle binary HDF output from FLO-2D. Several bugs related to FLO-2D file format were also resolved.

XDMF format

We are pleased to merge changes from Furgo GeoConsulting to support XDMF format. The new format supports on-the-fly loading of the data.

Sponsors

The new and improved support for file formats for HDF5 and UGRID have been kindly sponsored by FLO-2D and Deltares. We developed trace animation for fun.

Feedback and bug report

If you have any problem with Crayfish, please do not email us directly and consider filing a bug here: https://github.com/lutraconsulting/qgis-crayfish-plugin/issues

Crayfish 2.5: New Features

A new release of Crayfish is now available from QGIS plugin repo.

Here are the new features in more detail…

Trace animation

We have added support for live trace animation on QGIS canvas:

Please note that export to trace animation (as an avi file) is not supported yet. To change vector style to traces, please refer to the Crayfish manual.

Support for UGRID

UGRID format for Deltares modelling packages are now supported in Crayfish. In addition to 2D mesh, UGRID also supports 1D mesh. Support for 1D mesh is experimental in this release.

FLO-2D HDF format

In addition to ASCII FLO-2D files, Crayfish can handle binary HDF output from FLO-2D. Several bugs related to FLO-2D file format were also resolved.

XDMF format

We are pleased to merge changes from Furgo GeoConsulting to support XDMF format. The new format supports on-the-fly loading of the data.

Sponsors

The new and improved support for file formats for HDF5 and UGRID have been kindly sponsored by FLO-2D and Deltares. We developed trace animation for fun.

Feedback and bug report

If you have any problem with Crayfish, please do not email us directly and consider filing a bug here: https://github.com/lutraconsulting/qgis-crayfish-plugin/issues

WMTS Enhancement and XYZ Tile Native Support in QGIS 2.18

In this post, we will highlight the new features we have added to QGIS 2.18 …

WMTS enhancement

The WMTS provider had not been benefiting from the the QGIS multi-threaded rendering we did earlier in QGIS 2.4.

In previous versions of QGIS, users had to wait until download of all tiles of a layer has finished in order to view the resulting map. This has now been fixed and the tiles show up in map canvas immediately as they get downloaded, improving the user experience by greatly lowering the time until something is shown.

Moreover, previously downloaded tiles from lower or higher resolutions may be used for the preview functionality in the areas where the tiles with correct resolution have not been downloaded yet.

The screencast here shows fetching and rendering a WMTS layer in QGIS 2.14 (left) and the same layer in QGIS 2.18 (right):

Support for XYZ raster tiles

There are a couple of python plugins allowing users to add XYZ tiles (e.g. Bing maps) to QGIS. The plugins only allow certain web services and it is often tricky for supporting the private ones with API keys.

In addition, there are other QGIS applications without python support (e.g. QGIS for Android devices) where they can leverage from having a native support.

Currently, you can only add XYZ tile services from the Browser panel. The video below demonstrates how to add the current precipitation and OpenStreetMap xyz tiles to your QGIS:

Sponsors

WMTS enhancements was sponsored by Land Information New Zealand. Support for XYZ tiles was funded internally.

Editing Raster Cell Values in QGIS Using Serval Plugin

Users can directly edit raster cell values using Serval plugin in QGIS.

Read more for how to use this plugin…

How to use Serval

Serval is available from QGIS plugin repository. Note that you will need to restart QGIS if you upgrade Serval from an earlier version.

Once installed, Serval functions and settings will be available from the toolbar.

Serval Toolbar in QGIS

Serval supports Undo/Redo for editing values of raster. But it is recommended to make a copy of your raster.

Currently, the following functionalities are available:

  • Probe mode Displays raster bands values in boxes.
  • Draw mode Draw/Edit mode: bands values can be modified in the boxes and written to the current raster cell by hitting the Enter key. In this mode the values will be also assigned to any other raster cell clicked by user.
  • Write nodata To replace a cell value with the NODATA value.
  • Define nodata To define or replace the NODATA value.
  • Color picker To pick a color using QGIS color picker (3-bands rasters only).
  • Undo Redo To Undo/Redo the cell edit. Edits history is saved separately for each raster, that is, undo/redo is always done for current raster layer.

Future developments

We’d like to add support to edit values using spatial and expression selection tools.

For any problems or feedback, please consider to file a ticket here.

Crayfish 2.3: New Features

Crayfish 2.3 is out with FLO-2D format support and automatic export of contours

Here are the new features in more detail…

Support for new formats

We added support for FLO-2D result files.

FLO-2D format in Crayfish

Weather Research and Forecasting Model (WRF) outputs are now fully supported in Crayfish.

Export contours

Users can now directly generate vector contours from a Crayfish layer. The export-to-contour feature allows you to select type of contour (e.g. line or area contour) and contour intervals. A very handy option is that the current colour ramp can be also used for your contour intervals.

Exporting contours in Crayfish

Processing toolbox Crayfish provider

We have incorporated many algorithms from Crayfish to processing toolbox (Export grid, Export mesh elements, …). To activate the Crayfish module, from the main menu, select Processing > Options and in the new window, under Providers > Crayfish algorithms select the option for Activate.

Exporting contours in Crayfish

With the processing toolbox, user can create batch geo-processing algorithms to automate their work. For example, you can create a batch process using this module, to export several Crayfish grids to rasters.

Export to animation using the Processing Toolbox is not supported yet.

Future developments

We’d like to add support for mesh generation and also mesh calculator. If these are something of you or your organisation interest and would like to contribute financially, feel free to get in touch.

For any problems or feedback, please consider to file a ticket here.

Introducing First Aid Plugin

First Aid Plugin

Software development often consists of a brief period of pure excitement while writing the bulk of the source code, followed by a long and dreaded period of finding bugs and fixing them. This scheme is no different when writing plugins for QGIS.

Because we also write QGIS plugins ourselves, we were thinking of how to ease the pain of getting rid of bugs — and so we ended up creating the First Aid plugin, now available in the official QGIS plugin repository.

It is meant to be a Swiss army knife for QGIS plugin developers, a tool that allows easy inspection of any Python code running within QGIS. This is important because it can potentially save developers a lot of their valuable time.

How many times did you end up adding “print” statements into the code to find out what was going wrong in your code? With First Aid plugin this should be no longer necessary.

Error Handler

So let me explain what does it do. First of all, it comes with an improved Python error handler. What this means is that whenever an error occurs in code of a plugin, a window with all the details comes up.

Previously in QGIS you could only find out what was the exception’s type, message and stack trace. First Aid plugin adds source code view, variables view and even an embedded Python console where you can further inspect the state of the plugin at the time of the error. Here is how it looks like in action:

Error Handler Screenshot

Debugger

Plugins however do not always come up with errors that can be caught and handled by QGIS. More often plugins simply do not behave as one would expect them to. Here is when people usually resort to using debuggers. There are IDEs like PyDev or PyCharm – or even standalone tools like Winpdb – that allow developers to do remote debugging. Basically they can connect to the Python environment within QGIS and debug the code there. Personally, I have never been a big fan of this approach and found remote debugging cumbersome to set up and use. And trying to debug something on a client’s computer is even a greater challenge.

First Aid fortunately integrates a debugger into QGIS environment. This allows developers to simply open the debugger window, load some Python files, set breakpoints and everything is ready. Once QGIS reaches a line with a breakpoint, the debugger window will be activated and it is possible to step through the code and inspect the variables to understand what is going on in the code.

Debugger Screenshot

The great thing is that once the execution of Python code is stopped, it is possible to step into code, step over, step out or run to cursor, just like in any other debugger. It is also possible to run custom scripts from within debugger window — they will be also run in debug mode.

Debugging is active only while the debugger window is still open. While debugging, there is some extra overhead when running any Python code (even for code that you do not intend to debug), so it is better to close the debugger when not needed.

The plugin has already helped us various times to quickly identify problems in plugins. Having said that, please note that the plugin is still quite young and may not work perfectly in all cases. We would be happy to hear your feedback. Any issues or pull requests on GitHub would be greatly appreciated!

QGIS Code Sprint - FOSS4G UK - 2016

Issue Reporting

The Open Source community in the UK will get together for another great FOSS4G meeting from 14 to 16 of June in Southampton.

QGIS has already established itself as one of the major Open Source GIS applications. To help the community to contribute back to the project, we are running a code sprint.

The code sprint session will not require prior knowledge of coding, so all participants are welcome to join. Make sure you bring your laptop!

There will be several power users and plugin developers to help with the code sprint. There will be 3 topics covered during the code sprint:

  • Documentation
  • Helping with the bug fixing
  • Contributing to qgis2web plugin

Documentation

If you are interested in helping with the QGIS documentation, we will run a short presentation to introduce you to the workflow.

Participants are required to have installed and configured the following software packages:

Helping with the bug fixing

For those who are interested in findings bugs, testing existing bugs and leaning up the bug queue, you will need:

  • An OSGeo ID
    • Currently requires contacting OSGeo admins on IRC (#osgeo) in order to get the registration form (!)
  • Multiple versions of QGIS
    • QGIS LTR (LTS)
    • QGIS release
    • QGIS master

If you are familiar with coding (in C++ and python) and would like to fix some bugs, you will need:

  • QGIS development environment
  • Github account

Contributing to qgis2web plugin

qgis2web

Help improve qgis2web by joining any of these activities:

What you’ll need

  • laptop
  • QGIS, ideally current or nightly
  • a Github account
  • whatever Git/Github client you prefer
  • your favourite text editor

Details of some specific bugs and features are listed in a dedicated Github issue. If you are not coming to FOSS4GUK 2016, we’d still really value any help you can give. We’ll keep an eye on Github, or try the qgis2web Gitter.

This is a great opportunity to take your first steps in contributing to an open-source project. Come and help – we’ll tell you everything you need to know, and help you learn the skills you don’t yet have. Rest assured that we will be at least as grateful as you for the help you give.

If you are a beginner, @archaeogeek is also running a great workshop at FOSS4GUK 2016 on getting started in open-source: Don’t be afraid to commit

Using OS AddressBase for Address Search in QGIS

In this blog post we’ll learn how to use Ordnance Survey AddressBase data with the QGIS Discovery plugin for searching addresses.

Discovery Plugin for QGIS

Before we start

The AddressBase data will be loaded into a PostGIS table for Discovery to query. At this stage we should already have a functioning PostgreSQL / PostGIS installation.

A previous blog post describes how to quickly set up such an environment.

Creating the addressbase table

Let’s now create a table for storing the addressbase data. In the example below we’ll create a table called addressbase in the os_address schema.

The script below can be executed through pgAdminIII.

To run the script:

  1. Open pgAdminIII
  2. Connect to your destination database
  3. Select Query tool from the Tools menu
  4. Paste the code below into the Query tool
  5. Press F5 to execute the query (it may take a few seconds to complete)

When the query has finished you should see Query returned successfully with no result in … seconds. in the Messages panel:

pgAdminIII Messages Panel

At this point we should be able to locate the new addressbase table within the os_address schema:

addressbase Table

If you can’t see the schema / table you probably need to refresh the schemas / tables views in pgAdminIII’s Object browser panel by hitting F5.

``` — Create the destination schema if required CREATE SCHEMA IF NOT EXISTS os_address;

— Create a function which will populate the full_address and geom columns as — data are imported CREATE OR REPLACE FUNCTION create_geom_and_address() RETURNS trigger AS $$ BEGIN — The geometry — Set it based on the x_coord and y_coord fields NEW.geom = ST_SetSRID(ST_MakePoint(NEW.x_coordinate, NEW.y_coordinate), 27700); — The full address — Initialise it NEW.full_address = ‘’; — Build the full address by only including optional address components if they — exist IF NEW.organisation_name IS NOT NULL AND length(NEW.organisation_name) > 0 THEN

NEW.full_address = NEW.full_address || NEW.organisation_name || ', ';

END IF; IF NEW.department_name IS NOT NULL AND length(NEW.department_name) > 0 THEN

NEW.full_address = NEW.full_address || NEW.department_name || ', ';

END IF; IF NEW.po_box_number IS NOT NULL AND length(NEW.po_box_number) > 0 THEN

NEW.full_address = NEW.full_address || NEW.po_box_number || ', ';

END IF; IF NEW.sub_building_name IS NOT NULL AND length(NEW.sub_building_name) > 0 THEN

NEW.full_address = NEW.full_address || NEW.sub_building_name || ', ';

END IF; IF NEW.building_name IS NOT NULL AND length(NEW.building_name) > 0 THEN

NEW.full_address = NEW.full_address || NEW.building_name || ', ';

END IF; IF NEW.building_number IS NOT NULL THEN

NEW.full_address = NEW.full_address || NEW.building_number || ', ';

END IF; IF NEW.dependent_thoroughfare IS NOT NULL AND length(NEW.dependent_thoroughfare) > 0 THEN

NEW.full_address = NEW.full_address || NEW.dependent_thoroughfare || ', ';

END IF; IF NEW.thoroughfare IS NOT NULL AND length(NEW.thoroughfare) > 0 THEN

NEW.full_address = NEW.full_address || NEW.thoroughfare || ', ';

END IF;

NEW.full_address = NEW.full_address || NEW.post_town || ‘, ’;

IF NEW.double_dependent_locality IS NOT NULL AND length(NEW.double_dependent_locality) > 0 THEN

NEW.full_address = NEW.full_address || NEW.double_dependent_locality || ', ';

END IF; IF NEW.dependent_locality IS NOT NULL AND length(NEW.dependent_locality) > 0 THEN

NEW.full_address = NEW.full_address || NEW.dependent_locality || ', ';

END IF;

NEW.full_address = NEW.full_address || NEW.postcode;

RETURN NEW; END; $$ LANGUAGE ‘plpgsql’;

— Drop any existing addressbase table DROP TABLE IF EXISTS os_address.addressbase CASCADE; CREATE TABLE os_address.addressbase ( — id will be the primary key, populated automatically id serial NOT NULL, uprn bigint NOT NULL, os_address_toid varchar(24) NOT NULL, — os_address_toid bigint NOT NULL, udprn integer NOT NULL, organisation_name varchar(60), department_name varchar(60), po_box_number varchar(6), sub_building_name varchar(30), building_name varchar(50), building_number smallint, dependent_thoroughfare varchar(80), thoroughfare varchar(80), post_town varchar(30) NOT NULL, double_dependent_locality varchar(35), dependent_locality varchar(35), postcode varchar(8) NOT NULL, postcode_type char(1) NOT NULL, x_coordinate numeric(8,2) NOT NULL, y_coordinate numeric(9,2) NOT NULL, latitude numeric(9,7) NOT NULL, longitude numeric(8,7) NOT NULL, rpc char(1) NOT NULL, country char(1) NOT NULL, change_type char(1) NOT NULL, la_start_date date NOT NULL, rm_start_date date NOT NULL, last_update_date date NOT NULL, class char(1) NOT NULL, — the next two fields are populated automatically on insert full_address text NOT NULL, geom geometry(Point,27700) NOT NULL, CONSTRAINT addressbase_pkey PRIMARY KEY (id) ) WITH ( OIDS=FALSE );

— Create a pg_trgm index on the full_address column — This will allow super-fast, case-insensitive search on the column CREATE EXTENSION IF NOT EXISTS pg_trgm; CREATE INDEX addressbase_full_address_gin_trgm ON os_address.addressbase USING gin (“full_address” gin_trgm_ops);

— Spatial index for the geometry column CREATE INDEX addressbase_geom_gist ON os_address.addressbase USING gist (geom);

— trigger to create points and addresses — This trigger will be executed on each row inserted, calling the function defined above CREATE TRIGGER tr_create_geom_and_address BEFORE INSERT ON os_address.addressbase FOR EACH ROW EXECUTE PROCEDURE create_geom_and_address();

```

The script above has:

  • Created a table
  • Added any necessary indices
  • Created two additional, derived columns, full_address and geom

full_address will be used to store various address components into a sensible, human readable address. geom will be used to store point geometry based on address eastings/northings.

See the script comments for more information / detail.

Loading AddressBase

At this point we have an empty table ready to accept our AddressBase data. We will now import the data using pgAdminIII. Extract the CSV files for the addresses, you should end up seeing one or more CSV files, for example AddressBase_FULL_2016-03-19_001.csv

In pgAdminIII:

  1. Locate the addressbase table
  2. Right click it, select Import

An import dialog should appear. Select the first CSV file and set the settings in the File Options tab as shown here:

AddressBase Import Options 1

Uncheck the id, full_address and geom columns in the Columns tab as shown here:

AddressBase Import Options 2

Click Import. After a few seconds the dialog may report (Not Responding). This is nothing to worry about, be patient.

When the import process completes, close the import dialog and repeat the above steps with any remaining CSV files.

At this stage the data has been imported and the full_address field should contain sensible, human-readable addresses.

Configuring Discovery

With the data loaded in QGIS, we can now configure Discovery to make use of it.

  1. Install the Discovery plugin if not already installed
  2. Open Discovery’s settings using the button
  3. Set the settings as follows, changing the Scale Expression if required

Discovery Settings for OS AddressBase

Congratulations! QGIS should now be set up to search your AddressBase data.

  • Page 1 of 3 ( 48 posts )
  • >>

Back to Top

Sponsors