Tag: qgis

SLYR Update — June 2023

Welcome back, SLYR enthusiasts! We’re thrilled to share the latest updates and enhancements for our SLYR ESRI to QGIS Compatibility Suite that will dramatically streamline the use of ESRI documents within QGIS (and vice versa!). Our team has been hard at work, expanding the capabilities of SLYR to ensure seamless compatibility between the latest QGIS and ArcGIS releases. We’ve also got some exciting news for users of the Community Edition of SLYR! Let’s dive right in and explore the exciting new features that have been added to SLYR since our previous update

Converting Raster Layers in Geodatabases

We’re pleased to announce that SLYR now offers support for converting raster layers within Geodatabases. With this update, users can effortlessly migrate their raster data from ESRI’s Geodatabases to QGIS, enabling more efficient data management and analysis.

This enhancement is only possible thanks to work in the fantastic GDAL library which underpins QGIS’ raster data support. Please ensure that you have the latest version of QGIS (3.30.3 or 3.28.7 at a minimum) to make the most of this feature.

Annotation and Graphic Layer Improvements

Text Annotations along Curves

For those working with curved annotations, we’ve got you covered! SLYR now supports the conversion of text annotations along curves in QGIS. With this enhancement, you’ll get accurate conversion of any curved text and text-along-line annotations from MXD and APRX documents. This has been a long-requested feature which we can now introduce thanks to enhancements coming in QGIS 3.32.

ArcGIS Pro Graphics Layer Support

SLYR now supports the conversion of ArcGIS Pro graphics layers, converting all graphic elements to their QGIS “Annotation Layer” equivalents. If you’ve spent hours carefully designing cartographic markup on your maps, you can be sure that SLYR will allow you to re-use this work within QGIS!

Curved text graphic conversion

Enhanced Page Layout Support

We’ve further improved the results of converting ArcGIS Pro page layouts to QGIS print layouts, with dozens of refinements to the conversion results. The highlights here include:

  • Support for converting measured grids and graticules to QGIS map grids
  • Enhanced dynamic text conversions:  Now, when migrating your projects, you can expect a smoother transition for dynamic text ensuring your layouts correctly show generated metadata and text correctly
  • Support for north arrows, grouped elements, legends and table frames.

Rest assured that your carefully crafted map layouts will retain their visual appeal and functionality when transitioning to QGIS!

Improved QGIS to ArcGIS Pro Conversions

SVG Marker Exports and Symbology Size

SLYR has introduced initial support for exporting SVG markers from QGIS to ArcGIS Pro formats. SVG graphics are a component of QGIS’ cartography, and are frequently used to create custom marker symbols. Unfortunately, ArcGIS Pro doesn’t have any native support for SVG graphics for marker symbols, instead relying on a one-off conversion from SVG to multiple separate marker graphics whenever an SVG is imported into ArcGIS Pro. Accordingly, we’ve implemented a similar logic in SLYR in order to convert SVG graphics to ArcGIS Pro marker graphics transparently whenever QGIS symbology is exported to ArcGIS. This enhancement allows for a seamless transfer of symbology from QGIS, ensuring that your converted maps retain their visual integrity.

Furthermore, the update includes support for exporting QGIS symbology sizes based on “map unit” measurements to ArcGIS Pro, resulting in ArcGIS Pro symbology which more accurately matches the original QGIS versions.

Rule-Based Renderer Conversion

The “Rule Based Renderer” is QGIS’ ultimate powerhouse for advanced layer styling. It’s extremely flexible, thanks to its support for nested rules and filtering using QGIS expressions. However, this flexibility comes with a cost — there’s just no way to reproduce the same results within ArcGIS Pro’s symbology options! Newer SLYR releases will now attempt to work around this by implementing basic conversion of QGIS rule-based renderers to ArcGIS Pro layers with “display filters” attached. This allows us to convert some basic rule-based configuration to ArcGIS Pro formats.

There’s some limitations to be aware of:

  1. Only “flat” rule structures can be converted. It’s not possible to convert a nested rule structure into something representable by ArcGIS Pro.
  2. While the QGIS expression language is very rich and offers hundreds of functions for use in expressions, only basic QGIS filter expressions can be converted to ArcGIS Pro rules.

Improved Conversion of Raster and Mesh Layers

Based on user feedback, we’ve made significant improvements to the conversion of QGIS rasters and mesh layers to ArcGIS Pro formats. Expect enhanced accuracy when migrating these types of data, ensuring a closer match between your QGIS projects and their ArcGIS Pro equivalents.

New tools

The latest SLYR release introduces some brand new tools for automating your conversion workflows:

Convert LYR/LYRX Files Directly to SLD

To facilitate interoperability, SLYR has introduced algorithms that directly convert ESRI LYR or LYRX files to the “SLD” format (Styled Layer Descriptor). This feature simplifies the process of sharing and utilizing symbology between different GIS software, allowing for direct conversion of ESRI symbology for use in Geoserver or Mapserver.

Convert File Geodatabases to Geopackage

We’re thrilled to introduce a powerful new tool in SLYR that enables a comprehensive conversion of a File Geodatabase to the Geopackage format. With this feature, you can seamlessly migrate your data from ESRI’s File Geodatabase format to the versatile and widely supported GeoPackage format. As well as the raw data conversion, this tool also ensures the conversion of field domains and other advanced Geodatabase functionality to their GeoPackage equivalent, preserving valuable metadata and maintaining data integrity throughout the transition. (Please note that this tool requires QGIS version 3.28 or later.)

 

All these exciting additions to SLYR are available today to SLYR license holders. If you’re after one-click, accurate conversion of projects from ESRI to QGIS, contact us to discuss your licensing needs.

As described on our SLYR page, we also provide some of the conversion tools for free use via the SLYR “Community Edition”. We’re proud to announce that we’ve just hit the next milestone in the Community Edition funding, and will now be releasing all of SLYR’s support for raster LYR files to the community edition! This complements the existing support for vector LYR files and ESRI style files available in the community edition. For more details on the differences between the licensed and community editions, see our product comparison.

Learn More

[Blog] View and track changes in QGIS

With the recent changes to the Mergin Maps plugin for QGIS, you can visualise the local changes before synchronising your data. Read more in our blog post!
Learn More

How to contribute to QField

QField is a community-driven open-source project. It is free to share, use and modify and it will stay like that. The very essence of a community is to help and support each other. And that’s where YOU come into play. To make it work we need your support!

For those who don’t know much about the concept of open source projects, a bit of background. Investing in open-source projects is a technical and ethical decision for OPENGIS.ch. Open source is a technological advantage, as we receive input from many developers worldwide who are motivated to work out the best possible software. It prevents our customers from vendor lock-in and allows complete ownership and control of the developed software. And finally, not only financially independent businesses and people should benefit from professional software but also those who might not have the financial means to pay for features, and licences.

You are not a developer, but you still like to use QField and support it? Good news. You don’t have to be a developer to use, contribute or recommend the app. There are plenty of things that need to be done to help QField to remain the powerful software it is right now and become even better. Here are a few suggestions on how you can give something back.

  1. Review the app ★★★★★ in google’s play store or apple’s app store.
  2. Let the world know about it! It doesn’t matter if you’re on Twitter, LinkedIn, Instagram or any other social media platform. Show and tell about where QField helped you. We appreciate every post and we promise to like, share and comment.
  3. Write about your experience and please let us know. Be it in your blog or as a new success story. Insights into field projects are extremely valuable. It helps us to make the app even more efficient for your work, and it helps others to understand the range of applications for QField.
  4. Register for a paid QFieldCloud account. QFieldCloud allows to synchronize and merge the data collected in QField. QFieldCloud is hosted by the makers of QField and by getting an account you help QField too.

Do you want to do something that is more hands-on and directly linked to the app? No problem.

  1. Help with the documentation. You can document features, or improve the documentation in English. Read the how-to guide to get started.
  2. And if you are multilingual you might consider translating the documentation or the app in your language.
  3. Become a beta tester and be the first to report a bug! When something doesn’t work properly it might be a bug. The quicker we know about it, the faster it can be resolved.
  4. You can ask and answer questions on gis.stackexchange and help others on the user discussions platform.
  5. If you are a developer and you want to get involved in QField development, please refer to the individual documentation for QField, QFieldCloud and QFieldSync.

And now finally for those of you who have the financial means, you can either sponsor a feature or subscribe to one of the monthly sponsorships. By doing so you help get freshly baked QField versions straight to everyone’s devices.

Nothing in it for you? In that case, just drop by to say thank you or have a hot or cold beverage with us next time you meet OPENGIS.ch at a conference and you might make our day!
Want to know more about the idea of community-driven open-source projects and the QGIS project in particular? Check out Nyall Dawson’s blog post about how to effectively get things done in open source!

Learn More

QGIS 3.28 improvements for working with ESRI formats and services

The QGIS 3.28 release is an extremely exciting release for all users who work in mixed software workplaces, or who need to work alongside users of ESRI software. In this post we’ll be giving an overview of all the new tools and features introduced in 3.28 which together result in a dramatic improvement in the workflows and capabilities in working with ESRI based formats and services. Read on for the full details…!

Before we begin, we’d like to credit the following organisations for helping fund these developments in QGIS 3.28:

  • Naturstyrelsen, Denmark
  • Provincie Gelderland, Netherlands
  • Uppsala Universitet, Department of Archaeology and Ancient History
  • Gemeente Amsterdam
  • Provincie Zuid-Holland, Netherlands

FileGeodatabase (GDB) related improvements

The headline item here is that QGIS 3.28 introduces support for editing, managing and creating ESRI FileGeodatabases out of the box! While older QGIS releases offered some limited support for editing FileGeodatabase layers, this required the manual installation of a closed source ESRI SDK driver… which unfortunately resulted in other regressions in working with FileGeodatabases (such as poor layer loading speed and random crashes). Now, thanks to an incredible reverse engineering effort by the GDAL team, the open-source driver for FileGeodatabases offers full support for editing these datasets! This means all QGIS users have out-of-the-box access to a fully functional, high-performance read AND write GDB driver, no further action or trade-offs required.

Operations supported by the GDAL open source driver include:

  • Editing existing features, with full support for editing attributes and curved, 3D and measure-value geometries
  • Creating new features
  • Deleting features
  • Creating, adding and modifying attributes in an existing layer
  • Full support for reading and updating spatial indexes
  • Creating new indexes on attributes
  • “Repacking” layers, to reduce their size and improve performance
  • Creating new layers in an existing FileGeodatabase
  • Removing layers from FileGeodatabases
  • Creating completely new, empty FileGeodatabases
  • Creating and managing field domains

On the QGIS side, the improvements to the GDAL driver meant that we could easily expose feature editing support for FileGeodatabase layers for all QGIS users. While this is a huge step forward, especially for users in mixed software workplaces, we weren’t happy to rest there when we  had the opportunity to further improve GDB support within QGIS!

So in QGIS 3.28 we also introduced the following new functionality when working with FileGeodatabases:

FileGeodatabase management tools

QGIS 3.28 introduces a whole range of GUI based tools for managing FileGeodatabases. To create a brand new FileGeodatabase, you can now right click on a directory from the QGIS Browser panel and select New – ESRI FileGeodatabase:

After creating your new database, a right click on its entry will show a bunch of available options for managing the database. These include options for creating new tables, running arbitrary SQL commands, and database-level operations such as compacting the database:

You’re also able to directly import existing data into a FileGeodatabase by simply dragging and dropping layers onto the database!

Expanding out the GDB item will show a list of layers present in the database, and present options for managing the fields in those layers. Alongside field creation, you can also remove and rename existing fields.

Field domain handling

QGIS 3.28 also introduces a range of GUI tools for working with field domains inside FileGeodatabases. (GeoPackage users also share in the love here — these same tools are all available for working with field domains inside this standard format too!) Just right click on an existing FileGeodatabase (or GeoPackage) and select the “New Field Domain” option. Depending on the database format, you’ll be presented with a list of matching field domain types:

Once again, you’ll be guided through a user-friendly dialog allowing you to create your desired field domain!

After field domains have been created, they can be assigned to fields in the database by right-clicking on the field name and selecting “Set Field Domain”:

Field domains can also be viewed and managed by expanding out the “Field domains” option for each database.

Relationship discovery

Another exciting addition in QGIS 3.28 (and the underlying GDAL 3.6 release) is support for discovering database relationships in FileGeodatabases! (Once again, GeoPackage users also benefit from this, as we’ve implemented full support for GeoPackage relationships via the “Related Tables Extension“).

Expanding out a database containing any relationships will show a list of all discovered relationships:

(You can view the full description and details for any of these relationships by opening the QGIS Browser “Properties” panel).

Whenever QGIS 3.28 discovers relationships in the database, these related tables will automatically be added to your project whenever any of the layers which participate in the relationship are opened. This means that users get the full experience as designed for these databases without any manual configuration, and the relationships will “just work”!

Dataset Grouping

Lastly, we’ve improved the way layers from FileGeodatabases are shown in QGIS, so that layers are now grouped according to their original dataset groupings from the database structure:

Edit ArcGIS Online / Feature Service layers

While QGIS has had read-only support for viewing and working with the data in ArcGIS Online (AGOL) vector layers and ArcGIS Server “feature service” layers for many years, we’ve added support for editing these layers in QGIS 3.28. This allows you to take advantage of all of QGIS’ easy to use, powerful editing tools and directly edit the content in these layers from within your QGIS projects! You can freely create new features, delete features, and modify the shape and attributes of existing features (assuming that your user account on the ArcGIS service has these edit permissions granted, of course). This is an exciting addition for anyone who has to work often with content in ArcGIS services, and would prefer to directly manipulate these layers from within QGIS instead of the limited editing tools available on the AGOL/Portal platforms themselves.

This new functionality will be available immediately to users upon upgrading to QGIS 3.28 — any users who have been granted edit capabilities for the layers will see that the QGIS edit tools are all enabled and ready for use without any further configuration on the QGIS client side.

Filtering Feature Service layers

We’ve also had the opportunity to introduce filter/query support for Feature Service layers in QGIS 3.28. This is a huge performance improvement for users who need to work with a subset of a features from a large Feature Service layer. Unfortunately, due to the nature of the Feature Service protocol, these layers can often be slow to load and navigate on a client side. By setting a SQL filter to limit the features retrieved from the service the performance can be dramatically increased, as only matching features will ever be requested from the backend server. You can use any SQL query which conforms to the subset of SQL understood by ArcGIS servers (see the Feature Service documentation for examples of supported SQL queries).

 

What’s next?

While QGIS 3.28 is an extremely exciting release for any users who need to work alongside ESRI software, we aren’t content to rest here! The exciting news is that in QGIS 3.30 we’ll be introducing a GUI driven approach allowing users to create new relationships in their FileGeodatabase (and GeoPackage!) databases.

At North Road we’re always continuing to improve the cross-vendor experience for both ESRI and open-source users through our continued work on the QGIS desktop application and our SLYR conversion suite. If you’d like to chat to us about how we can help your workplace transition from a fully ESRI stack to a mixed or fully open-source stack, just contact us to discuss your needs.

Learn More

SLYR Update — November 2022

Our SLYR tool is the complete solution for full compatibility between ArcMap, ArcGIS Pro and QGIS. It offers a powerful suite of conversion tools for opening ESRI projects, styles and other documents directly within QGIS, and for exporting QGIS documents for use in ESRI software.

A lot has changed since our last SLYR product update post, and we’ve tons of very exciting improvements and news to share with you all! In this update we’ll explore some of the new tools we’ve added to SLYR, and discuss how these tools have drastically improved the capacity for users to migrate projects from the ESRI world to the open-source world (and vice versa).

ArcGIS Pro support

The headline item here is that SLYR now offers a powerful set of tools for working with the newer ArcGIS Pro document formats. Previously, SLYR offered support for the older ArcMap document types only (such as MXD, MXT, LYR, and PMF formats). Current SLYR versions now include tools for:

Directly opening ArcGIS Pro .lyrx files within QGIS

LYRX files can be dragged and dropped directly onto a QGIS window to add the layer to the current project. All the layer’s original styling and other properties will be automatically converted across, so the resultant layer will be an extremely close match to the original ArcGIS Pro layer! SLYR supports vector layers, raster layers, TIN layers, point cloud layers and vector tile layers. We take great pride in just how close the conversion results are to how these layers appear in ArcGIS Pro… in most cases you’ll find the results are nearly pixel perfect!

In addition to drag-and-drop import support, SLYR also adds support for showing .lyrx files directly in the integrated file browser, and also adds tools to the QGIS Processing Toolbox so that users can execute bulk conversion operations, or include document conversion in their models or custom scripts.

ArcGIS Pro map (mapx) and project (aprx) conversion

Alongside the LYRX support, we’ve also added support for the ArcGIS Pro .mapx and .aprx formats. Just like our existing .mxd conversion, you can now easily convert entire ArcGIS Pro maps for direct use within QGIS! SLYR supports both the older ArcGIS Pro 2.x project format and the newer 3.x formats.

Export from QGIS to ArcGIS Pro!

Yes, you read that correctly… SLYR now allows you to export QGIS documents into ArcGIS Pro formats! This is an extremely exciting development… for the first time ever QGIS users now have the capacity to export their work into formats which can be supplied directly to ESRI users. Current SLYR versions support conversion of map layers to .lyrx format, and exporting entire projects to the .mapx format. (We’ll be introducing support for direct QGIS to .aprx exports later this year.)

We’re so happy to finally provide an option for QGIS users to work alongside ArcGIS Pro users. This has long been a pain point for many organisations, and has even caused organisations to be ineligible to tender for jobs which they are otherwise fully qualified to do (when tenders require provision of data and maps in ArcGIS compatible formats).

ArcGIS Pro .stylx support

Alongside the other ArcGIS Pro documents, SLYR now has comprehensive support for reading and writing ArcGIS Pro .stylx databases. We’ve dedicated a ton of resources in ensuring that the conversion results (both from ArcGIS Pro to QGIS and from QGIS to ArcGIS Pro) are top-notch, and we even handle advanced ArcGIS Pro symbology options like symbol effects!

Take a look below how even very advanced ArcGIS Pro style libraries convert beautifully to QGIS symbol libraries:

ArcMap Improvements

While we’ve been focusing heavily on the newer ArcGIS Pro formats, we’ve also improved our support for the older ArcMap documents. In particular, SLYR now offers more options for converting ArcMap annotation layers and annotation classes to QGIS supported formats. Users can now convert Annotation layers and classes directly over to QGIS annotation layer or alternatively annotation classes can be converted over to the OGC standard GeoPackage format. When exporting annotation classes to GeoPackage the output database is automatically setup with default styling rules, so that the result can be opened directly in QGIS and will be immediately visualised to match the original annotation class.

Coming soon…

While all the above improvements are already available for all SLYR license holders, we’ve got many further improvements heading your way soon! For example, before the end of 2022 we’ll be releasing another large SLYR update which will introduce support for exporting QGIS projects directly to ArcGIS Pro .aprx documents. We’ve also got many enhancements planned which will further improve the quality of the converted documents. Keep an eye on this blog and our social media channels for more details as they are available…

You can read more about our SLYR tool at the product page, or contact us today to discuss licensing options for your organisation.

 

Learn More

24th Contributors QGIS Meeting in Firenze 2022

The international community of QGIS contributors got together in person from 18 to 22 August in parallel to OpenStreetMap State of The Map event and right before the FOSS4G. So there was a lot of open source geo power concentrated in the beautiful city of Florence in those days. It was my first participation and all I knew was that it’s supposed to be an unconference. This means, there is no strict schedule but space and opportunity for everyone to present their work or team up to discuss and hack on specific tasks to bring the QGIS project to the next level.

Introduction and first discussions

We were a group of six OPENGIS.ch members arriving mostly on Thursday, spending the day shopping and moving into our city apartment. In the evening we went to a Bisteccheria to eat the famous Fiorentina steak. It was big and delicious as was the food in general. Though, I am eating vegetarian since to compensate. On Friday we went to the Campus to meet the other contributors. After a warm welcome by the organizer, Rossella and our CEO and QGIS chair Marco Bernasocchi we did an introduction round where everyone mentioned their first QGIS version ever used. At this point, I became aware of the knowledge and experience I was sharing the room with. Besides this, I noticed that there was another company attending with several members, namely Tim Sutton’s Kartoza, which is also contributing a lot to QGIS. The first discussion was about QGIS funding model, vision, communication and on the new website in planning. This discussion then moved into some smaller groups including most of the long term contributors. I was looking around, physically and virtually, and tried to process all the new inputs and to better understand the whole QGIS world. Besides, I noticed my colleague Ivan having problems with compiling QGIS after upgrading to Ubuntu 22.04 which then motivated my other colleague Clemens to implement a docker container to do the compilation. Nevertheless, I postponed my Ubuntu upgrade. That evening we went out all together to have a beer or two and play some pool sessions and table football. Finally, the OPENGIS.ch crew navigated back home pairing a high-precision GNSS sensor with a mobile device running OpenStreetMap in QField. We arrived back home safely and super precise.

First tasks and coffee breaks

There was catering in the main hall covering breakfast, lunch and coffee breaks. It never took long after grabbing a cup of coffee to find yourself in a conversation with either fellow contributors or OpenStreetMap folks. I chatted with a mapper from Japan about mobile apps, an engineer from Colombia about travelling and a freelancer from the Netherlands about GDAL, to name 3 coffees out of many.

QGIS plugins website

After some coffee, Matthias Kuhn, our CTO and high-ranking QGIS contributor, asked me whether I could improve some ugly parts of QGIS plugins website. So I had my first task which I started working on immediately. The task was to make the site more useful on mobile devices which would be achieved by collapsing some unimportant information and even removing other parts. I noticed some quirks in the development workflow, so I also added some pre-commit hooks to the dev setup. Dimas Ciputra from Kartoza helped me finalize the improvements and merge them into master branch on github.

QGIS website downloads section

Regis Haubourg asked to help simplify the QGIS Downloads for Windows section on the main QGIS website. We played around in the browser dev tools until we thought the section looked about right. I then checked out the github repo and started implementing the changes. I need to say the tech stack is not quite easy to develop with currently, but there is a complete rework in planning. Anyway, following the pull request on github a lively discussion started which is ongoing by the time of writing. And this is a good thing and shows how much thought goes into this project.

Presentations

There were many interesting and sometimes spontaneous presentations which always involved lively discussions. Amy Burness from Kartoza presented new styling capabilities for QGIS, Tobias Schmetzer from the Bavarian Center for Applied Energy Research presented the geo data processing and pointed out issues he encountered using QGIS on this and Etienne Trimaille from 3liz talked about qgis-plugins-ci, just to name a few.

Amazing community

On Saturday evening a bus showed up at the campus and took us on a trip up to the hills. After quite a long ride we arrived at a restaurant high up with mind-blowing view of the city. I forgot how many rounds of Tuscan food were served, but it was delicious throughout. An amazing evening with fruitful conversations and many laughs.

The weather was nice and hot, the beers cold, the Tuscan food delicious and the contributors were not only popular Github avatars but really nice people. Thank you QGIS.

Learn More

A New Trick up QField’s Sleeve: Animated Maps

Starting with QField 2.2, users can fully rely on animation capabilities that have made their way into QGIS during its last development cycle. This can be a powerful mean to highlight key elements on a map that require special user attention.

The example below demonstrates a scenario where animated raster markers are used to highlight active fires within the visible map extent. Notice how the subtle fire animation helps draw viewers’ eyes to those important markers.

Animated raster markers is a new symbol layer type in QGIS 3.26 that was developed by Nyall Dawson. Supported image formats include GIF, WEBP, and APNG.

The second example below showcases more advanced animated symbology which relies on expressions to animate several symbol properties such as marker size, border width, and color opacity. While more complex than simply adding a GIF marker, the results achieved with data-defined properties animation can be very appealing and integrate perfectly with any type of project.

https://player.vimeo.com/video/732691644

You’ll quickly notice how smooth the animation runs. That is thanks to OPENGIS.ch’s own ninjas having spent time improving the map canvas element’s handling of layers constantly refreshing. This includes automatic skipping of frames on older devices so the app remains responsive.

Oh, we couldn’t help ourselves but take the opportunity to demonstrate how nice the QField feature form layout is these days in the video above ? To know more about other new features in QField 2.2, go and read the release page.

Happy field mapping to all!

The lovely animal markers used in the zoo example above were made by Serbian artist Arsenije Vujovic.

Learn More

Offline WMS - Benchmarking raster formats for QField

What are we looking for?

We would like to use WMS offline on QField. For that, we need to figure out what is the best way to get a raster from a WMS and which format is the most efficient (size and performance).

In this post we’ll show you is how to generate the ideal raster file from a WMS and the results of our efficiency tests for the the different raster formats.

WMS to GPKG

The simple way

If there is no limitation on the WMS or you need only a small region, here is the easiest process.

  1. Request the WMS and store a description file in XML:
gdal_translate "WMS:url" file.xml -of WMS
  1. Create a Geopackage from the information in the description file.
gdal_translate -of GPKG file.xml file.gpkg -co TILE_FORMAT=JPEG

That was quite simple, right?

The larger datasets way

If the command takes too much time, it means that it is trying to download too much data and could be caused by downloading higher resolution data than required.
The command might even completely fail if it contains a request for bigger data blocks thant the server allows.

Here is the process to get larger datasets in a simple way. Let’s use a real example:

  1. Use gdal_translate "WMS:https://www.gebco.net/data_and_products/gebco_web_services/web_map_service/mapserv?request=getmap&service=wms&crs=EPSG:4326&format=image/jpeg&layers=gebco_latest&version=1.1.0" test.xml -of WMS
  2. Open the test.xml file for editing, here you’ll find the parameters of the WMS. We change the “SizeX” to 3600 and “SizeY” to 1800. By changing these parameters we lower the resolution. It is important to keep proportionality.
  3. Another thing we need to change are “BlockSizeX” and “BlockSizeY” that define the size of the tiles. We change both to 2048.
  4. Finally, use gdal_translate -of GPKG test.xml test.gpkg -co TILE_FORMAT=JPEG
  5. To make a Geopackage pyramid use gdaladdo GPKG:test.gpkg:gebco_latest. It will replace the Geopackage, if you want to keep the original one, you need to copy it first.

Now you have a raster Geopackage that you can use in QField.

Testing raster formats

Preparing the files

As first step we exported our test orthophoto WMS to a plain GeoTIFF using QGIS’ default behaviour.

Formatgdal_translategdaladdogpkg JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_JPEG.gpkg” -co TILE_FORMAT=JPEG gpkg PNGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG.gpkg” -co TILE_FORMAT=PNGgpkg PNG_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG_JPEG.gpkg” -co TILE_FORMAT=PNG_JPEGgpkg PNG8gdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG8.gpkg” -co TILE_FORMAT=PNG8gpkg WEBPgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_WEBP.gpkg” -co TILE_FORMAT=WEBPgpkg pyramid_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_JPEG.gpkg” -co TILE_FORMAT=JPEGgdaladdo GPKG:C:\test\test_JPEG.gpkg:test_gpkg_JPEG gpkg pyramid_PNGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG.gpkg” -co TILE_FORMAT=PNGgdaladdo GPKG:C:\test\test_PNG.gpkg:test_gpkg_PNGgpkg pyramid_PNG_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG_JPEG.gpkg” -co TILE_FORMAT=PNG_JPEGgdaladdo GPKG:C:\test\test_PNG_JPEG.gpkg:test_gpkg_PNG_JPEGgpkg pyramid_PNG8gdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG8.gpkg” -co TILE_FORMAT=PNG8gdaladdo GPKG:C:\test\test_PNG8.gpkg:test_gpkg_PNG8gpkg pyramid_WEBPgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_WEBP.gpkg” -co TILE_FORMAT=WEBPgdaladdo GPKG:C:\test\test_WEBP.gpkg:test_gpkg_WEBPJPEG2000gdal_translate -of JP2OpenJPEG “C:\test\ortho_test.tif” “C:\test\test_jpeg_2000.jpg"COG DEFLATEgdal_translate “C:\test\ortho_test.tif” “C:\test\test_cog.tif” -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=DEFLATECOG_JPEGgdal_translate “C:\test\ortho_test.tif” “C:\test\test_cog_JPEG.tif” -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=JPEGtifIn QGIS right click on the layer > export > save as > (see the details in the picture under the table)MBTgdal_translate -of MBTILES “C:\test\ortho_test.tif” “C:\test\test_mbt.mbtiles"Creation commands for all the tested formats

Rendering test results

We have tested many formats, here is a table with the results of the size and rendering speed in QGIS and QField.
To analyze the speed we used qgis_bench.exe -i 10 -p "C:\test\test.qgs" >> "C:\test\test.log.
Qgis_bench is a tool that renders a QGIS project a number of times to get performance measurements. The parameter -i is to define the iterations and -p is the project used which contains only the generated raster.

FormatExtent [m]File size [GB]Total_avgTotal_maxdevTotal_minTotal_stdevgpkg JPEG52'880/29'2300.4250.242255.7815.539244.984gpkg PNG52'880/29'2302.9412.002490.328152.142259.859gpkg PNG_JPEG52'880/29'2300.4250.125256.8756.750245.172gpkg PNG852'880/29'2301.4283.875296.40612.625271.250gpkg WEBP52'880/29'2300.3330.238348.10973.534256.703gpkg pyramid_JPEG52'880/29'2300.51.0093.4062.3970.688gpkg pyramid_PNG52'880/29'2303.01.2083.2812.0730.688gpkg pyramid_PNG_JPEG52'880/29'2300.61.4914.3442.8531.016gpkg pyramid_PNG852'880/29'2301.61.5084.3752.8670.969gpkg pyramid_WEBP52'880/29'2300.41.3334.9063.5730.766JPEG200052'880/29'2301.113.888136.109122.2220.219COG DEFLATE52'880/29'2303.6264.427273.09425.411239.016COG_JPEG52'880/29'2301.014.778131.172116.3941.734tif52'880/29'2306.42.3676.7344.3671.672MBT52'880/29'2304.40.4694.6414.1710Comparison of file size and rendering speed of different raster formats. “Total” columns are rendering times in [s]. Lower file size is more storage friendly, lower Total_avg is more performant.

Analysis

File size

The Geopackage WEBP (with and without pyramid) has the best result for file size, but it is not _yet_supported by QField (from 1.6) and is only slightly smaller than the JPEG variant.

Plain GeoTiff, MBTiles, Cloud Optimized GeoTIFF (COG - DEFLATE mode) and Geopackages with PNG generate by far the largest file sizes (up to 20x larger) and are thus not recommended.

Rendering speed

MBTiles are on average double as fast as JPEG Geopackages with pyramids which in turn are more than double as fast as GeoTIFF and 15x faster than COG.
Geopackages without pyramids are 200 to 400 times slower.

Conclusion

Even though MBTiles render faster than the Geopackage pyramid JPEG, they come with an almost 10x bigger storage requirement which makes us say that the best offline raster format supported by QField is Geopackage pyramid JPEG or if you need transparency and slightly smaller files Geopackage pyramid WebP.

If you need transparency before QField 1.6, the best results are achieved with Geopackage pyramid PNG_JPEG.

Learn More

QField 1.4 released - Happy new year

What a year’s start! After a very packed December publishing all the QGIS on the road videos and quietly releasing QField 1.3 - Ben Nevis we could have gone and relaxed over the holidays. But since we love QField so much we immediately started working on the next iteration. Now, after an intensive testing period, we are proud to announce the release of QField 1.4 - Olavtoppen.

Olavtoppen!? yes, the highest point of Bouvet Island, the remotest island on Earth. And sure enough, QField would follow you there!

As usual, get it on play store or download it from GitHub.

QField Crowdfunding Campaign

Before digging into all the new goodness that you will find in QField 1.4, let’s get a big “Thanks” out to everybody who supported our crowdfunding campaign for improved camera support and all our customers that agreed to open source the work we did for them.

If you like QField, want a new feature or would like to support the project, don’t hesitate to get in touch with us.

Usability enhancements

In QField 1.2 we started to improve on the usability of the user interface. We are constantly working on this with a usability expert to get the user interface to be even more appealing and user-friendly.

Besides lots of clean-up and polishing, QField received two major improvements, a portrait mode and a new welcome screen with recent projects.

Welcome screen with recent projects

QField is all about efficiency. While favourites folders in the file selector already give a great productivity boost, very often we work with the same 3-4 projects. This is why we redesigned the welcome screen to list the last five project used. And if you look carefully you might get a hint of what will be coming soon…

Portrait mode

QField now flawlessly works in portrait mode. We heard you say you needed a comfortable way to work in portrait mode, especially on smartphones. QField forms and button placements are now optimized to be easy to use with your thumbs.

New features

We keep on listening to your feedback and prioritize new features based on it. We did implement some minor features like allowing hiding legend nodes and printing to PDF using the current extent. But this time’s superstars are three highly expected features: Splitting of geometries, compass integration and, yes you guessed right, native camera and gallery app support!

Split Features

ezgif com-optimize
ezgif com-optimize

A new editing tool is available that allows for splitting existing features. This adds an even more powerful operation to an already impressive geometry editing tools set.

Compass integration

A long-awaited feature! QField now shows you on-screen in which direction you are looking, walking, driving, flying or warping direction. This makes it much easier and more pleasant to navigate in the field.

Screenshot\_20200115-154223\_QField Nightly
Screenshot_20200115-154223_QField Nightly

It is now possible to use your favourite camera app so that you have more control over how pictures are taken. It is also possible to select pictures which are already on your device by using the new gallery selector.

Pro Tip: You can use any camera app. For example, you can use the open camera app to create geotagged photos if your preinstalled system camera doesn’t save positioning information in EXIF data.

Pro Tip 2: You can use an image annotation app to add notes, sketches, drawings and so on to your images and then choose them from QField via the add from gallery button.

Antenna Height Correction

For high precision measurements, it’s possible to compensate your altitude by a fixed antenna height. This will then automatically adjust all the digitised altitude values.

JPEG 2000

Support for JPEG 2000 raster datasets was added. This lossy format offers a compression rate at par with proprietary formats like ECW or Mr SID.

Pro Tip: save your base maps in JPEG 2000 to save storage.

New Languages

Thanks to the hard work of our community, QField is now also available in Turkish and Japanese.

New packages

You say: wow that’s a lot! We say: there is more :)
We have upgraded our whole building infrastructure so that you can comfortably get even more QField goodness without having to uninstall your production ready QField.

Automated master builds

After each pull request is merged into our master code, a new package is created and automatically published on the playstore in a dedicated app called QField for QGIS - Unstable (Early Access). Installing this app will allow you to always have the latest build of QField for testing and giving feedback. On your device, this app is completely separated from the production-ready QField and has a distinctive black icon so that you do not confuse it.

Pull request builds

QField is an extremely active project, and as you see we develop multiple functionalities and fixes at the same time. If you’re particularly interested in one of this, our continuous integration fairy builds and publishes new packages automatically at each commit directly to the pull request you are interested in. To see what we are currently working on, have a look at the pull request overview page.

Experimental Windows builds

Last but definitely not least, we’ve set up an Azure CI infrastructure to build QField for windows. For now, we still consider this experimental but we already had some very successful testing. If you are interested in testing out QField for windows you can get it here, remember it is experimental so don’t use it in production yet and give us as much feedback as possible :)

What’s next?

As you can imagine we’ve had a very busy start of 2020, but even more is to come soon with the next releases of QField. We’d like to thank again all companies and individuals that actively use QField and that invest in making QField even better. If you feel QField misses something you need or would like to support the project, don’t hesitate to get in touch with us.

Learn More