QGIS Planet

The QGIS 3.28 release is an extremely exciting release for all users who work in mixed software workplaces, or who need to work alongside users of ESRI software. In this post we’ll be giving an overview of all the new tools and features introduced in 3.28 which together result in a dramatic improvement in the workflows and capabilities in working with ESRI based formats and services. Read on for the full details…!

Before we begin, we’d like to credit the following organisations for helping fund these developments in QGIS 3.28:

• Naturstyrelsen, Denmark
• Provincie Gelderland, Netherlands
• Uppsala Universitet, Department of Archaeology and Ancient History
• Gemeente Amsterdam
• Provincie Zuid-Holland, Netherlands

FileGeodatabase (GDB) related improvements

The headline item here is that QGIS 3.28 introduces support for editing, managing and creating ESRI FileGeodatabases out of the box! While older QGIS releases offered some limited support for editing FileGeodatabase layers, this required the manual installation of a closed source ESRI SDK driver… which unfortunately resulted in other regressions in working with FileGeodatabases (such as poor layer loading speed and random crashes). Now, thanks to an incredible reverse engineering effort by the GDAL team, the open-source driver for FileGeodatabases offers full support for editing these datasets! This means all QGIS users have out-of-the-box access to a fully functional, high-performance read AND write GDB driver, no further action or trade-offs required.

Operations supported by the GDAL open source driver include:

• Editing existing features, with full support for editing attributes and curved, 3D and measure-value geometries
• Creating new features
• Deleting features
• Creating, adding and modifying attributes in an existing layer
• Full support for reading and updating spatial indexes
• Creating new indexes on attributes
• “Repacking” layers, to reduce their size and improve performance
• Creating new layers in an existing FileGeodatabase
• Removing layers from FileGeodatabases
• Creating completely new, empty FileGeodatabases
• Creating and managing field domains

On the QGIS side, the improvements to the GDAL driver meant that we could easily expose feature editing support for FileGeodatabase layers for all QGIS users. While this is a huge step forward, especially for users in mixed software workplaces, we weren’t happy to rest there when we  had the opportunity to further improve GDB support within QGIS!

So in QGIS 3.28 we also introduced the following new functionality when working with FileGeodatabases:

FileGeodatabase management tools

QGIS 3.28 introduces a whole range of GUI based tools for managing FileGeodatabases. To create a brand new FileGeodatabase, you can now right click on a directory from the QGIS Browser panel and select New – ESRI FileGeodatabase:

After creating your new database, a right click on its entry will show a bunch of available options for managing the database. These include options for creating new tables, running arbitrary SQL commands, and database-level operations such as compacting the database:

You’re also able to directly import existing data into a FileGeodatabase by simply dragging and dropping layers onto the database!

Expanding out the GDB item will show a list of layers present in the database, and present options for managing the fields in those layers. Alongside field creation, you can also remove and rename existing fields.

Field domain handling

QGIS 3.28 also introduces a range of GUI tools for working with field domains inside FileGeodatabases. (GeoPackage users also share in the love here — these same tools are all available for working with field domains inside this standard format too!) Just right click on an existing FileGeodatabase (or GeoPackage) and select the “New Field Domain” option. Depending on the database format, you’ll be presented with a list of matching field domain types:

Once again, you’ll be guided through a user-friendly dialog allowing you to create your desired field domain!

After field domains have been created, they can be assigned to fields in the database by right-clicking on the field name and selecting “Set Field Domain”:

Field domains can also be viewed and managed by expanding out the “Field domains” option for each database.

Relationship discovery

Another exciting addition in QGIS 3.28 (and the underlying GDAL 3.6 release) is support for discovering database relationships in FileGeodatabases! (Once again, GeoPackage users also benefit from this, as we’ve implemented full support for GeoPackage relationships via the “Related Tables Extension“).

Expanding out a database containing any relationships will show a list of all discovered relationships:

(You can view the full description and details for any of these relationships by opening the QGIS Browser “Properties” panel).

Whenever QGIS 3.28 discovers relationships in the database, these related tables will automatically be added to your project whenever any of the layers which participate in the relationship are opened. This means that users get the full experience as designed for these databases without any manual configuration, and the relationships will “just work”!

Dataset Grouping

Lastly, we’ve improved the way layers from FileGeodatabases are shown in QGIS, so that layers are now grouped according to their original dataset groupings from the database structure:

Edit ArcGIS Online / Feature Service layers

While QGIS has had read-only support for viewing and working with the data in ArcGIS Online (AGOL) vector layers and ArcGIS Server “feature service” layers for many years, we’ve added support for editing these layers in QGIS 3.28. This allows you to take advantage of all of QGIS’ easy to use, powerful editing tools and directly edit the content in these layers from within your QGIS projects! You can freely create new features, delete features, and modify the shape and attributes of existing features (assuming that your user account on the ArcGIS service has these edit permissions granted, of course). This is an exciting addition for anyone who has to work often with content in ArcGIS services, and would prefer to directly manipulate these layers from within QGIS instead of the limited editing tools available on the AGOL/Portal platforms themselves.

This new functionality will be available immediately to users upon upgrading to QGIS 3.28 — any users who have been granted edit capabilities for the layers will see that the QGIS edit tools are all enabled and ready for use without any further configuration on the QGIS client side.

Filtering Feature Service layers

We’ve also had the opportunity to introduce filter/query support for Feature Service layers in QGIS 3.28. This is a huge performance improvement for users who need to work with a subset of a features from a large Feature Service layer. Unfortunately, due to the nature of the Feature Service protocol, these layers can often be slow to load and navigate on a client side. By setting a SQL filter to limit the features retrieved from the service the performance can be dramatically increased, as only matching features will ever be requested from the backend server. You can use any SQL query which conforms to the subset of SQL understood by ArcGIS servers (see the Feature Service documentation for examples of supported SQL queries).

What’s next?

While QGIS 3.28 is an extremely exciting release for any users who need to work alongside ESRI software, we aren’t content to rest here! The exciting news is that in QGIS 3.30 we’ll be introducing a GUI driven approach allowing users to create new relationships in their FileGeodatabase (and GeoPackage!) databases.

At North Road we’re always continuing to improve the cross-vendor experience for both ESRI and open-source users through our continued work on the QGIS desktop application and our SLYR conversion suite. If you’d like to chat to us about how we can help your workplace transition from a fully ESRI stack to a mixed or fully open-source stack, just contact us to discuss your needs.

Our ninjas have been so busy that less than a month after we released QField 2.4, we find ourselves with so many new features we simply can’t wait any longer to present to you the latest version of QField: 2.5 “Fancy Flamingo ?”.

Exciting new features

QField’s main new feature of this 2.5 release cycle is its brand new elevation profiling functionality which has been added to the measuring tool. Users are now able to dynamically build and analyze elevation profiles wherever they are - in the field or on their desktop - by simply drawing paths onto their maps and projects.

This is a great example of QField’s capability at bringing the power of QGIS through a UI that keeps things simple and avoids being in your way until you need it. Oh and while we’re speaking of the measuring tool, check out the new azimuth measurement!

This new version also brings multi-column support to feature forms. QField now respects the number of columns set by users in the attributes’ drag and drop designer while building and tweaking projects in QGIS. The implementation will take into account the screen availability and on narrow devices will revert to a one-column setup. Pro tip: try to change the background color of your individual groups to ease understanding of the overall feature form.

Another highlight of this release is a brand new screen lock action that can be triggered through QField’s main menu found in the side dashboard or in the map canvas menu shown when long pressing on the map itself. Once activated, QField will become unresponsive to touch and mouse events while keeping the display turned on. When locked, QField also hides tool buttons which results in a more complete view of the map extent.

Stability improvements

As with every release, our ninjas have been spending time hunting nasty bugs and improving stability and QField 2.5 is no exception. In particular, the feature form should feel more reliable and even more polished.

Our SLYR tool is the complete solution for full compatibility between ArcMap, ArcGIS Pro and QGIS. It offers a powerful suite of conversion tools for opening ESRI projects, styles and other documents directly within QGIS, and for exporting QGIS documents for use in ESRI software.

A lot has changed since our last SLYR product update post, and we’ve tons of very exciting improvements and news to share with you all! In this update we’ll explore some of the new tools we’ve added to SLYR, and discuss how these tools have drastically improved the capacity for users to migrate projects from the ESRI world to the open-source world (and vice versa).

ArcGIS Pro support

The headline item here is that SLYR now offers a powerful set of tools for working with the newer ArcGIS Pro document formats. Previously, SLYR offered support for the older ArcMap document types only (such as MXD, MXT, LYR, and PMF formats). Current SLYR versions now include tools for:

Directly opening ArcGIS Pro .lyrx files within QGIS

LYRX files can be dragged and dropped directly onto a QGIS window to add the layer to the current project. All the layer’s original styling and other properties will be automatically converted across, so the resultant layer will be an extremely close match to the original ArcGIS Pro layer! SLYR supports vector layers, raster layers, TIN layers, point cloud layers and vector tile layers. We take great pride in just how close the conversion results are to how these layers appear in ArcGIS Pro… in most cases you’ll find the results are nearly pixel perfect!

In addition to drag-and-drop import support, SLYR also adds support for showing .lyrx files directly in the integrated file browser, and also adds tools to the QGIS Processing Toolbox so that users can execute bulk conversion operations, or include document conversion in their models or custom scripts.

ArcGIS Pro map (mapx) and project (aprx) conversion

Alongside the LYRX support, we’ve also added support for the ArcGIS Pro .mapx and .aprx formats. Just like our existing .mxd conversion, you can now easily convert entire ArcGIS Pro maps for direct use within QGIS! SLYR supports both the older ArcGIS Pro 2.x project format and the newer 3.x formats.

Export from QGIS to ArcGIS Pro!

Yes, you read that correctly… SLYR now allows you to export QGIS documents into ArcGIS Pro formats! This is an extremely exciting development… for the first time ever QGIS users now have the capacity to export their work into formats which can be supplied directly to ESRI users. Current SLYR versions support conversion of map layers to .lyrx format, and exporting entire projects to the .mapx format. (We’ll be introducing support for direct QGIS to .aprx exports later this year.)

We’re so happy to finally provide an option for QGIS users to work alongside ArcGIS Pro users. This has long been a pain point for many organisations, and has even caused organisations to be ineligible to tender for jobs which they are otherwise fully qualified to do (when tenders require provision of data and maps in ArcGIS compatible formats).

ArcGIS Pro .stylx support

Alongside the other ArcGIS Pro documents, SLYR now has comprehensive support for reading and writing ArcGIS Pro .stylx databases. We’ve dedicated a ton of resources in ensuring that the conversion results (both from ArcGIS Pro to QGIS and from QGIS to ArcGIS Pro) are top-notch, and we even handle advanced ArcGIS Pro symbology options like symbol effects!

Take a look below how even very advanced ArcGIS Pro style libraries convert beautifully to QGIS symbol libraries:

ArcMap Improvements

While we’ve been focusing heavily on the newer ArcGIS Pro formats, we’ve also improved our support for the older ArcMap documents. In particular, SLYR now offers more options for converting ArcMap annotation layers and annotation classes to QGIS supported formats. Users can now convert Annotation layers and classes directly over to QGIS annotation layer or alternatively annotation classes can be converted over to the OGC standard GeoPackage format. When exporting annotation classes to GeoPackage the output database is automatically setup with default styling rules, so that the result can be opened directly in QGIS and will be immediately visualised to match the original annotation class.

Coming soon…

While all the above improvements are already available for all SLYR license holders, we’ve got many further improvements heading your way soon! For example, before the end of 2022 we’ll be releasing another large SLYR update which will introduce support for exporting QGIS projects directly to ArcGIS Pro .aprx documents. We’ve also got many enhancements planned which will further improve the quality of the converted documents. Keep an eye on this blog and our social media channels for more details as they are available…

You can read more about our SLYR tool at the product page, or contact us today to discuss licensing options for your organisation.

What a night it was. The “Best of Swiss Apps Awards” took place in Zurich yesterday, November 2, 2022. We were also nominated with QField . And in the enterprise category, the app was so convincing, that it was awarded the highest possible price. So it brought the award “Best of Swiss Enterprise App” home to Graubünden. And as cherry on the cake: QField was also nominated as finalist in the UX/UI category!

We are extremely proud and happy about the received award. And even more when we look at the contendants that won in the other categories. We’re talking companies like SBB, Swiss Life, Switzerland Tourism and, yes, Rivella ?.

You can check out all results at https://www.bestofswissapps.ch/bosa/hall-of-fame

If you are interested in more details, we released a press release in German and in English.

QField is an open source mobile app. The app is designed to use and edit geographically referenced data. In urban environments with 5G connectivity, but also with offline data. The mobile GIS app combines minimal design for simplicity with sophisticated technology for a versatile range of uses to bring data conveniently from the field to the offices. The app was started in 2011 and received a major rebuild in 2022.

QField is mainly funded by customer feature requests, support contracts and sponsoring and is continuously improved an released for Android, iOS, Windows, MacOS and Linux.

It offers a seamless QGIS integration and is GPS-centric, with offline functionality, synchronisation options and desktop configuration. QField is designed for fieldwork: simple, but uncompromising. The app is used internationally and is the first choice for mobile GIS projects. In the city, in the countryside and in the forest.

Soon, QFieldCloud will also be launched. QFieldCloud is a cloud service integrated into QField that enables the remote provision and synchronisation of geodata and projects.

https://videopress.com/v/c95KEqlR?resizeToParent=true&cover=true&preloadContent=metadata&useAverageColor=true

And here some moments of the award night. It was a blast!

Yes, QField for QGIS, the leading fieldwork app, was released on the iOS App Store!

Get It now for Android, iOS, MacOS, Windows and Linux

Good things take time (and sponsors ), and we wanted our Apple users to enjoy the same solid and seamless experience as our Android users. So we took the time needed and ran beta testing of QField for multiple months. Thanks to all the community feedback and to the uncountable work hours put in by our development team, today we released QField on the iOS Appstore.

Following the naming scheme for the 2.X series, we decided to name QField 2.4 Ecstatic Elk (Cervus Canadensis), honouring “the land of maple leaf ??”, the home country of Mathieu (QField lead UX designer) and origin of some recent funding.

New features, improvements and demo projects

Releasing for iOS is the main news for QField 2.4, but we also added some new features as well as fixed some annoying bugs we had.

The new features include atlas-driven print layouts that can now be printed through the main menu’s print to PDF item and dragging files onto an iOS device via USB Cable with iTunes support.

Some more UX improvements can be noticed when sending or exporting datasets via the project folder. All sidecars will now be considered so that, for example, you can send your edited shapefiles via your favourite email or messenger app.

Finally, QFieldCloud’s projects are better sorted, and its community tab is now functional.

Bugfixes

During the last sprint, we greatly improved QField’s automated testing framework, greatly decreasing the risks of regressions slipping into future releases. We also ensured that QGIS-shipped SVG markers will now render properly within QField.

Finally, we fixed freehand toggling when using a stylus and ensured the changelog popup doesn’t overlap with the OS’ status bar.

Best of Swiss Apps Nomination

We put a lot of effort into ensuring that QField, is of the highest possible quality, so being nominated as a finalist for the BestOfSwissApps award was even sweeter ???

Beginning of November, we’ll know more about the outcome of the votes ?

QFieldCloud

QFieldCloud has been in Free BETA for half a year now. Thanks to the precious help of the many early adopters (we already have over 30K users), we were able to identify and fix plenty of issues. In the last months, our service status page has been consistently looking super-green ;)

We are extremely happy with how the system is behaving and are even happier with the feedback we are receiving!

As of today, we’ve implemented all the functionality that we want to have for the GA release. All we are missing is finishing testing the billing and payment system and rolling the drums ?

So keep on enjoying our fantastic fieldwork ecosystem, and let us know the amazing things you do with it!

Support QField

We put a lot of effort into ensuring that QField, is of the highest possible quality and invest a lot of developer time in QField, QFieldCloud and QGIS. Plenty of it is sponsored by OPENGIS.ch because we believe in giving back to the OS geo community; part is sponsored by the clients that ask us to develop features , and part is financed through our support contracts that come with a sustainability initiative .

If you think that helping QField is a good thing, go to our donate page to find out more or immediately start sponsoring QField .

In the past year, the build system behind QField has been ported to vcpkg, a modern C++ dependency management system . It has been a great success for QField and considerably helped to streamline efforts, improve the development experience and to guarantee an outstanding stability of the application. In this blog post we will look at the history of building QGIS based applications for mobile systems and how it has become what it is today.

When Marco Bernasocchi (CEO of OPENGIS.ch and chair of QGIS.org) started working on QGIS for Android in Google Summer of Code a decade ago, the main job was to also build all QGIS dependencies for Android. This includes well-known libraries like proj and gdal and less-known ones like libxml2 or iconv. Each of them has its particularities and specific build flags. Working on this appears to be an endless iterative trial-and-error journey where you hope each day that eventually you will see the QGIS splash screen on your Android phone while all you see are endless lines of code and compiler errors.

As we know nowadays QGIS for Android has eventually seen the sunlight and its achievements are still the base for QGIS-based mobile apps like QField .

Sometime later we decided to modernize the build infrastructure into OSGeo4A a set of scripts where each dependency was built with a “recipe”. Modularized this way, it was easier to maintain, and general build code common for all libraries could be isolated. It was good enough to help drive QField for a couple of years, and a copy of it is still in use as the base for nowadays QGIS builds for macOS .

When we decided to make QField also available on other platforms like iOS, Windows and macOS we quickly realized that duplicating build chains scales really bad and maintaining this is an immense effort we wanted to avoid. There are a couple of existing C++ dependency management systems, none of which convinced us ultimately. Lucky for us a mail on the QGIS mailing list mentioned a new one called vcpkg which looked very promising.

A couple of days later we had a build for Windows and later in the same year for macOS. With many dependencies already available in modern versions. Cheers.

What’s left to do than just enable it for Android, and all our problems are suddenly solved? Alas, it’s not so easy. Cross-compiling is always a bit trickier. And so we started another journey to improve the situation. After a while, we had a working build chain based on vcpkg for Android in our R&D labs. Interestingly, this added a couple of features just because the community around vcpkg had already added them. For example using COG -based raster data via HTTP was suddenly working (for the record: thanks to the availability of curl which we never took care of adding ourselves in OSGeo4A).

Soon after we also wanted to try building for iOS with vcpkg, which after a few attempts also was successful, and even managed to resolve some weird crashes and other issues we had experienced with the old buildchain.

The main benefit was that we could upgrade the QGIS base libraries in one single place for every platform, in an isolated branch without playing the Jenga game on each upgrade.

The only unfinished business we still had was that support for iOS and Android was still available only in our own vcpkg fork.

So the last few weeks and months we have been working closely with upstream to bring building for Android and iOS up to the same level as desktop platforms. The relevant parts are now in a clean state.

Advantages of this approach:

• • Mutualized efforts on all the base libraries, also with programmers outside the geoverse
• • A vibrant community that ensures a noticeably fast upgrade of libraries
• • A clean dependency management system
• • A consistent set of dependency versions (gdal, geos, libpq, …) across all platforms
• • A clean caching system that will only recompile reverse dependencies on updates
• • We can upgrade a dependency in an isolated branch and only release it when it works on all platforms
• • We can optimize the code for a given set of dependency versions and if a bug is fixed in a certain dependency version, we are sure we can ship this fix on all platforms promptly
• • We maintain the QField source code as well as dependency versions in a single repository, what makes development more streamlined

Big thanks go to Alexander Neumann and Kai Pastor who both stand out for doing things the right and future-proof way.

As always, things come at a price, there was a steep learning curve involved, and some edge cases require attention. However, we are thrilled by the simplification this has brought us.

If you are maintaining a customized fork of QField, it is now a good time to start upgrading to vcpkg, since OSGeo4A has been archived and will no longer be maintained. The developer documentation of QField has been updated with relevant instructions.

If you have time to test the new build system, we will be happy to read about your experiences with it.

The international community of QGIS contributors got together in person from 18 to 22 August in parallel to OpenStreetMap State of The Map event and right before the FOSS4G . So there was a lot of open source geo power concentrated in the beautiful city of Florence in those days. It was my first participation and all I knew was that it’s supposed to be an unconference. This means, there is no strict schedule but space and opportunity for everyone to present their work or team up to discuss and hack on specific tasks to bring the QGIS project to the next level.

Introduction and first discussions

We were a group of six OPENGIS.ch members arriving mostly on Thursday, spending the day shopping and moving into our city apartment. In the evening we went to a Bisteccheria to eat the famous Fiorentina steak. It was big and delicious as was the food in general. Though, I am eating vegetarian since to compensate. On Friday we went to the Campus to meet the other contributors. After a warm welcome by the organizer, Rossella and our CEO and QGIS chair Marco Bernasocchi we did an introduction round where everyone mentioned their first QGIS version ever used. At this point, I became aware of the knowledge and experience I was sharing the room with. Besides this, I noticed that there was another company attending with several members, namely Tim Sutton’s Kartoza, which is also contributing a lot to QGIS. The first discussion was about QGIS funding model, vision, communication and on the new website in planning. This discussion then moved into some smaller groups including most of the long term contributors. I was looking around, physically and virtually, and tried to process all the new inputs and to better understand the whole QGIS world. Besides, I noticed my colleague Ivan having problems with compiling QGIS after upgrading to Ubuntu 22.04 which then motivated my other colleague Clemens to implement a docker container to do the compilation. Nevertheless, I postponed my Ubuntu upgrade. That evening we went out all together to have a beer or two and play some pool sessions and table football. Finally, the OPENGIS.ch crew navigated back home pairing a high-precision GNSS sensor with a mobile device running OpenStreetMap in QField . We arrived back home safely and super precise.

First tasks and coffee breaks

There was catering in the main hall covering breakfast, lunch and coffee breaks. It never took long after grabbing a cup of coffee to find yourself in a conversation with either fellow contributors or OpenStreetMap folks. I chatted with a mapper from Japan about mobile apps, an engineer from Colombia about travelling and a freelancer from the Netherlands about GDAL, to name 3 coffees out of many.

QGIS plugins website

After some coffee, Matthias Kuhn, our CTO and high-ranking QGIS contributor, asked me whether I could improve some ugly parts of QGIS plugins website . So I had my first task which I started working on immediately. The task was to make the site more useful on mobile devices which would be achieved by collapsing some unimportant information and even removing other parts. I noticed some quirks in the development workflow, so I also added some pre-commit hooks to the dev setup. Dimas Ciputra from Kartoza helped me finalize the improvements and merge them into master branch on github .

QGIS website downloads section

Regis Haubourg asked to help simplify the QGIS Downloads for Windows section on the main QGIS website. We played around in the browser dev tools until we thought the section looked about right. I then checked out the github repo and started implementing the changes. I need to say the tech stack is not quite easy to develop with currently, but there is a complete rework in planning. Anyway, following the pull request on github a lively discussion started which is ongoing by the time of writing. And this is a good thing and shows how much thought goes into this project.

Presentations

There were many interesting and sometimes spontaneous presentations which always involved lively discussions. Amy Burness from Kartoza presented new styling capabilities for QGIS, Tobias Schmetzer from the Bavarian Center for Applied Energy Research presented the geo data processing and pointed out issues he encountered using QGIS on this and Etienne Trimaille from 3liz talked about qgis-plugins-ci , just to name a few.

Amazing community

On Saturday evening a bus showed up at the campus and took us on a trip up to the hills. After quite a long ride we arrived at a restaurant high up with mind-blowing view of the city. I forgot how many rounds of Tuscan food were served, but it was delicious throughout. An amazing evening with fruitful conversations and many laughs.

The weather was nice and hot, the beers cold, the Tuscan food delicious and the contributors were not only popular Github avatars but really nice people. Thank you QGIS.

Starting with QField 2.2, users can fully rely on animation capabilities that have made their way into QGIS during its last development cycle. This can be a powerful mean to highlight key elements on a map that require special user attention.

The example below demonstrates a scenario where animated raster markers are used to highlight active fires within the visible map extent. Notice how the subtle fire animation helps draw viewers’ eyes to those important markers.

Animated raster markers is a new symbol layer type in QGIS 3.26 that was developed by Nyall Dawson . Supported image formats include GIF, WEBP, and APNG.

The second example below showcases more advanced animated symbology which relies on expressions to animate several symbol properties such as marker size, border width, and color opacity. While more complex than simply adding a GIF marker, the results achieved with data-defined properties animation can be very appealing and integrate perfectly with any type of project.

https://player.vimeo.com/video/732691644

You’ll quickly notice how smooth the animation runs. That is thanks to OPENGIS.ch’s own ninjas having spent time improving the map canvas element’s handling of layers constantly refreshing. This includes automatic skipping of frames on older devices so the app remains responsive.

Oh, we couldn’t help ourselves but take the opportunity to demonstrate how nice the QField feature form layout is these days in the video above ? To know more about other new features in QField 2.2, go and read the release page .

Happy field mapping to all!

The lovely animal markers used in the zoo example above were made by Serbian artist Arsenije Vujovic .

TLDR: Since November 2021, Google has enforced new storage access limitations for apps published on its Play store which prohibits direct storage access on Android 11 and above forcing QField to adapt and rely on importing projects and datasets to access those.

If you are a QField beta user on Android 11 and above, you might have noticed a significant change in the way the app is handling storage in the latest set of betas released in early February of 2022. This blog post will go over the changes, explain why those had to be made (looking at you, Google), how to work in this new paradigm, and showcase some new benefits from the hard work done by OPENGIS.ch’s geoninjas.

It’s all gone! How can I access my projects and datasets?!

Starting with Android 11, apps are denied full access to main and external storage content. For QField, this means direct access to projects and datasets transferred and/or downloaded into storage folders is not possible anymore.

To work within this new confine, QField now has to import project folders or individual datasets into an app-dedicated storage location where Android allows for unrestricted read/write access.

Practically, this means that instead of being shown and having access to the full storage tree when clicking on the “Open local files” button, users are now shown a set of new folders named ‘QField files directory’, ‘Imported datasets’, and ‘Imported projects’ as well as a drop-down menu accessible via a top-right three-dot button.

Import project from folder

When importing a project from a folder, users will be asked to grant permission for QField to read the content of a given folder on the device’s storage via a system folder picker. When the folder is selected, QField copies the folder content (including its sub-folders) into the app’s ‘Imported projects’ location. Users can then open the project from there.

Re-importing a given folder through the drop-down menu action will overwrite preexisting projects given an identical folder name. That allows users to be able to update projects.

Note that feature editing, addition, and deletion will be saved into the imported project’s datasets, not in the original folder selected during the import process. More on how to find and handle those project datasets will come later in this post.

Import project from ZIP archive

Having to adapt to Google’s new set of rules did not come without its benefits. Users can now easily transfer projects into a given device by compressing the project content into a ZIP archive and having QField import that compressed project automatically. This can greatly ease remote deployment of projects by being able to send a single file to users.

Import dataset(s)

QField can also import individual dataset(s). Users will be asked to select one or more files via a system file picker, which will be copied into the ‘Imported datasets’ folder. Users will have to ensure that all sidecar files are selected when importing (e.g. a shapefile dataset would require users to select the .shp, .shx, .dbf, .prj, and .cpg files).

Just like imported projects, editing of datasets will be saved into the imported datasets, and not reflected in the original files.

Alright, but how can I retrieve modified projects and datasets?

Imported projects and datasets can be accessed directly using a USB cable. The location on storage is displayed in the top navigation bar when opening a local file.

On most devices plugged into a computer via USB cable connection, the path will be :/Android/data/ch.opengis.qfield/files/ where you will find both the Imported Datasets and Imported Projects folders within which your edited content will be located.

However, we’ve also added a nice new ‘Send to…’ functionality that allows for users to share and send datasets straight from QField using Android APIs. This allows for the sending of edited datasets directly to third party apps (Gmail, Drive, Dropbox, Nextcloud, your favourite messenger app, etc.).

Is direct copying via USB cable gone altogether?

Users can still avoid going through the import process by copying files via a USB cable connection directly into the QField app’s files directory. As mentioned above, the location on most devices will be :/Android/data/ch.opengis.qfield/files/.

What are the benefits from these changes?

Working out a functional solution to meet Google’s newly-enforced restrictions did not come without its benefits.

On top of what was already covered above - importing of compressed project ZIP files and sharing functionalities – QField is now fully integrated with Android’s cross-application document sharing APIs. This means that users can now directly open projects and files sent to them via their favourite browser/email/cloud/messenger app without the need to first download those files onto the device.

Altogether, the newly-coded importing mechanisms and integration with Android document APIs don’t only improve the ease of use for the average person, it also makes viewing and editing spatial datasets on QField far more secure. The imported projects and datasets reside in a location with access limited to QField only, meaning that its content is inherently far more protected from malicious access from third-party apps.

Why were these drastic changes needed?

As mentioned in the introduction, the changes were needed to comply with a set of new Google Play policies that came into force in November 2021. Users can read more on Google’s rationale on this page https://developer.android.com/google/play/requirements/target-sdk .

As part of the enforcement of these new policies, Google came up with an arbitrary mechanism to whitelist some apps which allows those to retain full storage access given the user explicitly allowed for it. We here at OPENGIS.ch believes QField had ample justifications to be whitelisted, however, Google’s appeal process judged otherwise after a series of email exchanges detailing our reasoning. While we have so far lost this argument with Google, we will continue fighting for our users and for their freedom to choose. If by any chance you have a good contact at Google that might be willing to listen to our reasoning, we would be grateful if you’d get in touch with us .

We hope this clarifies the recent changes and helps QField users adapt to those.

For once, it’s not an app from the Silicon Valley, but from Laax in the Swiss Alps that made the news. By publishing QField as an open-source app, OPENGIS.ch allows companies, organisations and even countries without the necessary financial means to have the opportunity to benefit from this important data collection app. And it is being used: Over half a million downloads have already been achieved. Now, since the volcanic eruption in Tonga on 15 January 2022, the app of the small Laax-based company is playing a not-unimportant role in disaster response planning.

We’ve only been around for seven years. We could almost pass for a start-up. But OPENGIS.ch is already a household name when it comes to field data collection. OPENGIS.ch makes its field mapping tool QField available without restrictions (i.e. open-source) so that companies or even nations can collect their geographical data. In this way, data is brought from the field to the office and provides important insights for future decisions.

This was also the case for Tonga’s volcanic eruption. There, in December 2021, many relevant agricultural datasets were captured with “QField” and “QFieldCloud”. This currently helps the Ministry of Agriculture, Food, Forests, and Fisheries to analyse the damage after the volcanic eruption and the subsequent tsunami and plan the response.

Swiss software in use for Tonga

It is a technical and ethical decision for me. I want not only the privileged nations to benefit from our work.

Marco Bernasocchi, CEO

Co-founder Marco Bernasocchi’s credo is to focus on further developing the product (QField). OPENGIS.ch makes it freely available, so that nations like Tonga, with little financial means, can use professional software and invest their financial resources in reconstruction. “It is a technical and ethical decision for me. I want not only the privileged nations to benefit from our work. We are always developing the product and generate income mainly from support, paid developments and custom solutions. Our product, however, is publicly accessible”. This strategy is successful: the company is 90% financed by Swiss customers but the app is used all over the world. “Open source is a technological advantage for me. This way we get the input of many developers worldwide who are motivated to work out the best possible software. This leads to a superior product and is particularly valuable to me.,” adds Matthias Kuhn, co-founder of OPENGIS.ch.

This way we get the input of many developers worldwide who are motivated to work out the best possible software.

Matthias Kuhn, CTO

John Duncan, project manager at the University of Western Australia, who is working on the project in support of Tonga, explains the work done in December like this: " the workflow uses QField to map the extent of landscape features including agricultural fields and cropped holdings, recording detailed attributes about each feature’s farming system. QFieldCloud was used to support large teams of data collectors tasked with surveying every tax allotment across Tonga. The initiative has resulted in the detailed mapping of cropping and livestock systems for over 11,000 farms across Tonga’s three main island groups. These datasets provide actionable information for short-term decision making around food security, agricultural planning, and disaster response, and are a valuable resource for longer-term monitoring of agricultural and environmental changes in these climate-vulnerable locations."

The data was originally intended for food security and agricultural planning. But now they suddenly have acquired enormous value and can be used for disaster response planning.

John Duncan, University of Western Australia

Further information on QField in connection with Tonga:

• https://livelihoods-and-landscapes.com/use-case.html
• https://www.aciar.gov.au/project/asem-2016-101

Media articles

• https://www.suedostschweiz.ch/aus-dem-leben/buendner-software-im-einsatz-fuer-tonga
• https://www.inside-it.ch/buendner-open-source-app-nuetzt-der-katastrophenhilfe-in-tonga
• https://epaper.somedia.ch/issue.act?issueMutation=wzbw&issueDate=20220302#Redaktionell
• https://www.chilitalk.ch/schweiz/technologie/katastrophenhilfe-aus-laax/
• https://www.business-geomatics.com/2022/02/03/katastrophenhilfe-fuer-tonga-mit-open-source-loesungen-qfield-und-qfieldcloud/
• https://www.presseportal-schweiz.ch/pressemeldungen/weltweit-fuehrendes-buendner-unternehmen-liefert-wichtige-feld-datenerfassungs-app

About OPENGIS.ch

OPENGIS.ch GmbH is a Swiss software development company based in Laax. OPENGIS.ch employs 19 people and works mainly in the field of spatial software development, geodata infrastructure deployments and professional support. Personalised open source GIS solutions are often planned and developed as desktop or mobile applications. OPENGIS.ch finances itself through tailor-made customer solutions, professional support and adaptations. Link: https://opengis.ch

About the OPENGIS.ch product “QField” application

“QField” is an open-source app developed for efficient fieldwork in real-time in urban areas, with 5G connection or with offline data. The mobile GIS app combines a minimal design with sophisticated technology to conveniently bring data from the field to the office. Seamless QGIS integration, GPS centred, offline functionality, synchronisation capabilities, desktop configurable: “QField” is designed for fieldwork - simple but uncompromising. Link: https://qfield.org

About the OPENGIS.ch service “QFieldCloud”

“QFieldCloud” is a spatial cloud service integrated into “QField” that allows remote provisioning and synchronisation of geodata and projects. Although “QFieldCloud” is still in an advanced beta stage, it is already being used by many groups to significantly improve their workflows. Link: https://qfield.cloud