QGIS Planet

Today, on QField’s 10th anniversary, we’re extremely proud to publish the results of over 18 months of development and give you the source code of QFieldCloud to go and make your awesome adaptations, solutions, and hopefully contributions :)

If you want to quickly try it out, head to https://qfield.cloud where our hosted solution is running and secure yourself a spot in the beta program.

QFieldCloud’s unique technology allows your team to focus on what’s important, making sure you efficiently get the best field data possible. Thanks to the tight integration with the leading GIS fieldwork app QField, your team will be able to start surveying and digitising data in no time.

What a journey it was and what plans do we already have… It has now been 10 years since I pushed the first scripts to build Quantum GIS for Android and it is incredible what we’ve been able to achieve thanks to a vibrant community, sponsors and especially our fantastic team .

At OPENGIS.ch  we strongly believe in giving back . We live from open-source projects and are deeply committed to sustaining their technological and economic well-being . We also believe everyone should have access to the best possible tools and knowledge. By committing ourselves to develop open-source applications, we give everyone access to powerful tools to plan, review and mitigate geospatial issues.

That is why we are even more thrilled to have created and open-sourced a professional data and team management solution for the best QGIS fieldwork app and would like to share a bit of the history of how we revolutionised field work by creating QField for QGIS.

Prehistory - QGIS for Android is born

Stone-, bronze-, iron-age, you get it, the beginnings of field mapping in the QGIS world were pretty rough around the edges. It all started thanks to me being accepted in the Google Summer of Code 2011 programme with the “QGIS mobile” submission . In the following 3 months, I’d try, with the help of my mentors Pirmin Kalberer and Marco Hugentobler, to get Quantum GIS to run on my tablet.

The first start

Hi all, it is a pleasure to announce that I finally got Quantum GIS to start on an android (3.2) tablet (Asus transformer). I tested as well on a Samsung Galaxy phone with cyanogen mod 7 RC1 and it works well (with the obvious screen size limitations).
Qgis still doesn’t load many elements, but the GUI is there and the rest should be only minor issues. I’ll post more as soon as I make further developments. Meanwhile, if you want to test the apk, you can download it from my GitHub here . For building your own, have a look at qgis wiki

/2011/08/17/qgis-on-android/

https://vimeo.com/27793965

The first ever video about QGIS on Android

A proper GUI

See my last posts. In short, I managed to get qgis packaged as an APK and to properly run with only one major problem. The map canvas is always black. I’ll investigate this till Tuesday.
Cheers

/2011/08/18/qgis-on-android-has-a-proper-gui/

https://vimeo.com/27854857

After 3 months of intensive work, QGIS for android finally has a a proper GUI

Blazing fast startup

Hi, I just managed to create an APK with all the resources needed by qgis …

The only inconvenience at the moment is that at the first startup the app shows a black screen while it’s copying the files for about 30 to 60sec so just be patient and remember that the whole app will take up to 230MB (it installs on external storage by default)

/2011/08/19/qgis-on-android-has-complete-gui-and-supports-translations/

A working reality

I still remember the feeling that day when after almost 3 months, of fighting with shell scripts, patching of build systems, debugging via ADB, writing C++ in Java wrappers and so on, my Quantum GIS test project was suddenly running on my tablet… I Was so happy I just went running in the mountains :).

Just a quick screenshot to show that qgis on android is now a working reality. Tomorrow I’ll make a video and so on. The major missing thing now is reading SHP files ad maybe spatialite… maybe tomorrow. Now it’s Sunday ?

/2011/08/21/qgis-android-works-2/

GSoC 2011 results

At the end of the Google Summer of code, I received my MSc in geoinformatics and left for 3 Months to Indonesia working as a consultant/developer for the World-bank Global Facility for Disaster Reduction and Recovery.

So, it is over, after 3 months of working on QGIS for android as a Google Summer of Code project it is now time to wrap up what I did and didn’t do.
First of all a QGIS android app exists now and it has many features including:
– reading/writing projects
– raster support
– spatialite support
– WMS support
– (apparent – untested) WFS and Postgres support
– partial shape files support (string attributes still crash the app)
– Fully functional GUI (SymbologyV2 doesn’t work yet)
– (all?) core C++ plugins beside globe (any takers? ?)
Furthermore, I created a series of build scripts that make it easier to set up a dev environment.
Unfortunately, I didn’t manage to implement live GPS tracking and a larger GUI optimisation, but all in all, I’m very happy with the results and seeing that few peoples are already testing it. Soon ill publish a video.
cheers

/2011/08/24/gsoc-2011-final-report/

Quantum GIS for Android was a reality and I was fully committed to keeping working on it. Turns out I wasn’t wrong :)

Classical - QGIS for Android grows

The Next Era of QGIS for android is what could be seen as the time of great knowledge enhancement, philosophical musings and the rise of the first great features including:

• GPS support including external GPS
• Compass support
• Right-click support
• Pinch zooming, tap zooming and panning
• armeabi-v7a optimizations
• the 3.2" screen experiment
• Extreme environment testing

https://vimeo.com/36862461

Middle Ages - QGIS Mobile

The dark ages, times of instability, change and some setbacks. Sounds terrifying, it was not at all, on the contrary it was a very formative period that apexed with the fantastic release of QGIS 2.0 for android.

The QML app experiment

From the beginning on, the idea behind QGIS for android was to eventually ditch the GUI and build a dedicated one for touch devices. The Google Summer of code 2012 by Ramon Carrillo mentored by myself set off to do that. Unfortunately, the project encountered some roadblocks and never took off as expected, but it did lay some ideas and code for the future.

The Python failure

Probably the major setback in QGIS for android’s history was the non-completion of the Python support. I got really close to it multiple times but unfortunately never managed to tame the snake. Maybe something we’ll look into in future, who knows.

/2013/05/20/python-support-in-qgis-is-getting-there/

/2013/05/21/getting-closer-to-taming-the-snake/

/2013/05/21/python-suport-even-closer/

The QGIS 2.0 release

The pivotal point of the Middle Ages was definitely 20.09.2013, when Tim Sutton presented to a full auditorium the shiny new QGIS 2.0. And along with it it introduced the general availability of QGIS 2.0 on android. The first real QGIS version for mobile devices was finally available for the broad public.

After the launch followed a very active time of keeping QGIS for Android on pair with the desktop versions leading to a regular release of updates on the playstore between 2013 and late 2014. This is also when Matthias Kuhn started committing to the QGIS for Android repository .

https://vimeo.com/75261402

Early Modern - QField for QGIS is here

Humanism, Renaissance and Enlightenment are what we saw happening in the period between 2015 and early 2019. Field users were put at the centre of the design process, new ideas were explored and a new name was chosen to reflect the main goal of the application: Make fieldwork as efficient as possible.

Early 2015 was also when Matthias Kuhn and myself decided to join forces in OPENGIS.ch LLC.

The rebranding

The project never had a clear name, at times it was called QGIS for Android at times QGIS mobile, we felt that to clearly convey what we were building we needed a clear, simple and poignant name.

It is with great pleasure that we want to announce the new name for what was briefly known as QGIS mobile.

Please welcome QField for QGIS™!

After long thinking about various names and variants including QGIS mobile, QTouch, OPENGIS.ch QGIS mobile, QWork, and many more, we felt that QField represents best what we want to archive. A field data capture and management app fully compatible with QGIS™.

/2015/01/28/qgis-mobile-is-now-qfield/

https://vimeo.com/116231850

QField Experimental is out, after a couple of months of requirements gathering, private early alpha testing and foremost tons of  emails requesting access to the testes group we decided today to put the current BETA version in the playstore.

/2015/06/15/qfield-in-the-wild/

Streamlined installation

Since the beginning of QGIS for android, to distribute the needed Qt libraries, we used a project called “ministro”. This was interesting because it allowed to download the libraries only once but on the other hand, it was a very painful experience for the user that needed to install a second app before getting QField to start. Around the end of 2015 it was so far, we finally managed to get rid of that dependency and make the installation process as streamlined as possible.

It’s done, finally we managed to get rid of Ministro so that we finally can say, QField runs on any android from 4.0.3 (ICS). This makes as of today (according to google ) 96% of the android installations worldwide.

/2015/12/01/qfield-for-android-5/

Release candidates

In these 3.5 years, a continuous, mainly volunteer-driven iterative process led by Matthias Kuhn made QField grow to the point where we felt confident it was time to launch QField 1.0.

After a series of release candidates with lots of feedback from the community, we felt it was time to move into a New Era.

It was a long and winding road but we are very excited to announce the general availability of QField 1.0 Release Candidate 1.

Packed with loads of useful features like online and offline features digitizing, geometry and attributes editing, attribute search, powerful forms, theme switching, GPS support, camera integration and much more, QField is a powerful tool for those who need to edit on the go and would like to avoid standing in the swamp with a laptop or paper charts.

/2019/01/08/qfield-1-0-rc1/

Modern times - QField 1.X

Fast forward to March 28th 2019,

Let’s get straight to the point

It’s official, QField for QGIS 1.0 is out!

Get it while it’s hot on the Playstore ( qfield.org/get ) or on GitHub

We are incredibly pleased and proud of just having released such a jewel and are convinced that thanks to all its features and conscious design choices, QField will make your field digitizing work much more efficient and pleasant.

/2019/03/28/qfield-1-0-is-here/

Recent releases

In the last 2 years the development pace increased tremendously, the sponsored featured grew as never before, QField rating skyrocketed to 4.7 ⭐ we currently have 100'000 active users and we’re getting around 500 new users every day.

Our QField core team grew more and now thanks to David Signer’s lead and Mathieu Pellerin keen eye for UX we’re pushing QField even further.

Going into all releases would be so much information that this post would turn into a 3 volumes classic, and since starting from QField 1.0 we’ve documented each new release, we’re just going to link them: /category/qfield/highlights/

The future is cloudy - ehm sunny of course ;)

Yesterday we published QField 1.9.6, which is going to be the last 1.X release and will put QField 2.0 into the beta channel so that every beta tester can start using QFieldCloud without having to use the developer version .

But that is a different story and you can read all about it in our latest newsletter …

Get fieldwork smoothly and nimbly done despite the ice and snow outside. Collect accurate data with freehand digitizing and improved form widgets, use the data from your external GNSS receivers without any third-party apps and enjoy the pleasant usability of QField 1.8 Selma.

This year started off hi-speed for us. There’s been already a lot of coding, designing and teaching, and we’ve thrown ourselves into these things we love to do. And we published another QField release last week that I completely forgot to announce in this blog. But here it is. It’s QField 1.8, Selma. And it’s packed with cool features.

Let’s have a look.

Freehand drawing

This might be a feature that brings a lot of fun and professionalism to your work. The freehand digitizing mode allows the user to “draw” lines and polygons with the stylus pen. The mode is available for adding line/polygon features as well as for the ring tool of the geometry editor.

Together with the powerful options in the topological editing where you can snap to existing features and avoid overlaps, it’s very convenient to digitize complex shapes.

Zoom in and out

Speaking of fun. One day, a guy from the QGIS community asked us if we could implement the functionality to zoom in and zoom out like he is able to do with an app called Maps from a company named Google. I didn’t know what he meant, but he explained: Single finger double tap-and-hold zoom gesture (which allows you to zoom smoothly from anywhere on the screen). Wow! Didn’t know it before, but it’s super neat! So we made it available in QField as well.

If you are used to it, it’s quite easy. But for beginners it can be a bit difficult. So for people who are not that deft - and to keep the UX self-explanatory and simple - we also added two buttons + / - to zoom in and zoom out with just one finger. So now even a clumsy pirate with a hook instead of a hand can collect data with QField :-)

Powerful Relation Reference Widget

Let’s be a little bit more serious and talk about how powerful the relation reference widget has become.

View and Edit selected feature

The intuitive eye icon next to the widget lets you open the form of the referenced parent feature to view and edit it.

Autocomplete mode

When auto-complete is enabled, you can easily perform a search in all available parent features.

With space-separated input, you can search for the beginning of multiple words in the display name of the parent features. So in this example searching for “Ma” will find the name “Mae” and “Marie” and using the second word “buck” it finds the Buckfast bees - so the entries containing both values will be listed on top.

Integration of external GNSS receivers

In case you wondered, why we did not release 1.8 Selma earlier? Because we wanted to have it feature loaded and rocket proof. And one of this cool feature is the integration of external GNSS receivers.

QField can receive and decode NMEA sentences received via Bluetooth from an external GNSS receiver (such as an EMLID Reach RS2) without the need for any third party app.

Search for paired Bluetooth devices in the device settings, connect to the external device and receive the GNSS information.

Select vertical grid shift files

In the QField settings, you can select a grid file on your mobile device by placing it in a directory named QField/proj in the main folder of the internal storage to increase the vertical location accuracy.

Postgres Config File

If you once started using PostgreSQL configuration files, you don’t want to live without them anymore. And when you use it on your PC, I’m sure you want to use it on your mobile device as well.

Define Postgresql services in a pg_service.conf file and use it on QField by placing it directly in a directory named QField in the main folder of the internal storage.

Add reload data button

The layer properties have been polished and in addition, you will find a button to reload the layer data. This is especially useful if you use WFS layers from which you need to get updates.

Register extra fonts

Also, you can add TTF and OTF font files into a directory named QField/fonts at the main folder of the internal storage to use the nice fonts you like.

How beautiful is that!

Support of new raster file formats

By the way: Many new raster file formats are supported - most notably COG . While not yet supported as remote format streamed directly from the web, it is also a high performance format if used locally

What about the cloud?

You might be one of these people eagerly waiting and always receiving the same message: Keep calm, it’s coming soon. Sorry for that. But when we do something, we do it right. And we prefer to have a stable solution than to publish half baked stuff. We are still highly busy coding, testing and promoting QFieldCloud . It’s announced for this spring / early summer.

Also, keep an eye on the @QFieldForQgis and @QFieldCloud twitter accounts to stay updated.

We ♥ our Beta Testers

The Beta Testers are our secret heroes. They report bugs and inconveniences before the normal users are bothered with them. Thanks to the Beta Testers QField is so stable. And at this point we would like to say: Thank you, test heroes!

And what do the beta testers get in return? Well, they can be the very first to try out the great new features. This is exciting and fun. So don’t hesitate. Join the beta.

In the Play Store you should find this section under the “QField for QGIS” app listing. Enjoy the feature frenzy and report the problems at qfield.org/issues

And if you wondered…

… why this release is called “Selma”. It’s of course because of the Mount Selma in Australia… And because it’s the name of my beloved cat. That’s her - Selma Eulenkopf - staring at me while I’m coding QField.

Be ready for the cold weather with a smooth coordinate search, filters in the value relation widget, fancy new QML and HTML widgets, enhanced geometry editing functionalities and an expandable legend. Right when Autumn starts, QField 1.7 Rockies hits the stage.

As usual get it now on the play store or on github !

The days are getting shorter and the wind blows colder. It’s always good to be in a good company outside while getting your mapping work done. QField will be your reliable companion.

We know, QField 1.6 Qinling has only been out two months and with its amount of new features and stability improvements, it would have deserved a longer primetime. But we just couldn’t withhold you all the new great stuff we’ve been building lately.

So let’s welcome QField 1.7 Rockies. And yes, we mean THE Rockies , where QField is looking for plenty of new buddies.

Let’s have a look.

Merging features

Splitting of a feature has been possible for quite some time. Now the merging of features of multipolygon-layer is possible as well. Select them and merge them - easy like that. The first selected feature gets the new geometry and keeps its attributes.

Filters in the Value Relation Widget

The value relation widgets provide an easy selection of a related feature. Often it’s used for lookup tables but sometimes the related tables contain a lot of entries and the list of the possible values is long.

Using filters in the value relation drop-down can increase the efficiency in selecting the correct value. It can be configured by expressions in QGIS, so it’s possible to have the content of the drop down depend on the values entered previously in other fields.

In the screenshot above there is a Map Value Widget with “forest” and “meadow” as values. On selecting “forest”, only the trees appear in the Field “Plant Species”. On selecting “meadow” there would be listed flowers instead.

Go to coordinates in the Search

The search has not only been improved in its appearance, but it’s handling is much more comfortable with a button to clear the text and easy opening and closing.

Additionally, we added the possibility to jump to coordinates. Searching a place you know the coordinates of is now super simple. And this means that digitizing that precise geometry with known coordinates is finally possible.

QML and HTML Widget

You might remember when we introduced the QML widget in QGIS . Now it’s in QField as well. And it’s not alone. HTML widgets are supported too.

This provides a lot of possibilities to display information with texts, images and charts and it even allows you interaction.
Do you need help setting up complex forms? Don’t hesitate to get in touch with us !

Expandable legend icons

The legend items are now expandable and collapsible.

Wait a minute… Wasn’t this possible before? Yes. It was possible in earlier versions. But why it’s announced here as a new feature?

Because now it is built in a future proof manner thanks to all the people and organisations who care for QField and bought a support contract with the sustainability initiative  or committed to a recurring sponsorship .

Some technical background: As you may be aware QField uses QGIS under the hood and QGIS uses Qt under the hood. Qt is currently used in version 5. Qt 5 is not that young any more and has a lot of functionality which is no longer supported by Qt. The old legend was based on the tree view, a deprecated module. Using it had some implications like the suboptimal support of HiDPI. Furthermore, these deprecated modules will disappear in the soon-to-come Qt 6.

As you can see, keeping QField at the quality we and you expect requires a lot of maintenance work. It is of utmost importance and only possible thanks to sponsoring since paying for fixing already existing features is less attractive for most people.

What will the future bring

In the last weeks, we have been highly busy on coding, testing and promoting QFieldCloud and we are very happy to be able to announce it very soon. So be prepared.

Also, keep an eye on the @QFieldForQgis and @QFieldCloud twitter accounts to stay updated.

Open Source

QField is an open source project. This means that whatever is produced is available free of charge. To anyone. Forever. This also means that everyone has the chance to contribute. You can write code, but you don’t need to. You can also help translating the app to your language or help out writing documentation or case studies or by sponsoring a new feature.

And now…

… enjoy QField 1.7 Rockies and have a nice autumn!

Editing multiple features at the same time, support for stylus pens, dynamic configuration of image names and much more.
QField 1.6 Qinling 秦岭 comes packed with awesome new features and an improved user experience.

We have been very busy over the last few months working on a new and shiny QField release. We have added many new features that increase efficiency on the field or allow for new workflows. In parallel, we have also been working on ironing out a series of issues and improving the overall user experience to make the app as pleasurable to use as possible. The result is QField 1.6 which has been published now.

Enough of the highlevel talking, let’s see what has been done.

Multi editing

Do you recall Geography lesson 101, Toblers first law? Everything is related to everything else. But near things are more related than distant things.

Very often there are similar objects nearby which share a property, tree species tend to group, human created objects like street light types or street paint markings tend to be of the same type at the same location.

With QField 1.6 it is now much easier to select a couple of features and change an attribute with very few taps. Identify a feature, long press an identify results, select more features and click the edit attributes button.

Stylus support

Sometimes it is just too cold to be working with fingers (although of course you can get capacitive gloves too). Or you just prefer to be working with a pen. QField 1.6 comes with support for stylus pens. If your device ships with one, give it a try.

Lock geometries

For some scenarios, especially in asset management, you only need to change attributes of existing objects and never add new features, delete features or change geometries. This can be configured through QFieldSync and set in the layer properties.

Image name configuration

Did you ever want to have the file names of your pictures to match the feature id, the layer name or any free text? The expression based configuration in QFieldSync offers now complete freedom in naming your images.

Legend and UX and legacy code

Didn’t expect to read UX and legacy code in one single title?

QML is the technology on which the QField user interface is built. QML ships a lot of user interface elements in a library called “Quick Controls”. A long time ago already it received an update from version 1 to version 2. Up to recently we still have been using some elements from version 1, which had an effect on high resolution displays not being able to properly display everything. To workaround that we introduced a lot of band aids, to improve the situation. We are very happy, that by migrating the legend and few other remaining elements to Quick Controls 2 in version 1.6, we have been able to completely drop this code.

Topological editing

QGIS can detect shared boundary by the features, so you only have to move a common vertex once, and QGIS will take care of updating the neighboring ones. So does his little college QField since this release.

Fast editing mode

For the real adventurers who know what they are doing this release brings the fast editing mode. In this mode, the features will automatically be stored on every change. The user interface is lighter and it combines perfectly with the topological editing.

Unter the hood

We have brought the whole technology stack up to speed with modern requirements. Proj and GDAL have been updated to recent versions. This helped to mitigate a couple of issues with coordinate transformations that were completely misplaced . It also paves the path for a future with datum corrections and always more important high precision measurements.

Known Issues

Unfortunately, we are experiencing a crash on startup with 32 bit devices. These devices are not that common any more, but if you have a device that is already a couple of years old it’s very well possible that it comes with a 32 bit cpu builtin. Despite the team’s hard efforts to isolate the reason, we were not able to find out what it was yet. Because of this we will not be able to update to 1.6 for these devices at the moment. We still hope that we will find a solution for this but don’t know yet when this will be.

We have updated proj to version 6. This brings plenty of bug fixes with coordinate handling. Among other things it adds support for using datum grids (gsb files) for very precise transformations, it is not yet possible to install those on the device. You will get an information message in the about dialog if your project happens to fall into this category. In this case, as a workaround switch the CRS of the project to a CRS with a known conversion that works without grid files.

What will the future bring

You guessed it already, we are not tired and have plenty of things stacked for the future. Prepare for more exciting updates for attribute forms and also for QFieldCloud which is right now being tested in our R&D labs.

Also keep an eye on the @QFieldForQgis Twitter account to stay updated.

Open Source

QField is an open source project. This means that whatever is produced is available free of charge. To anyone. Forever. This also means that everyone has the chance to contribute. You can write code, but you don’t need to. You can also help translating the app to your language or help out writing documentation or case studies or by sponsoring a new feature.

Thanks to sponsors

Various organisations have helped to make this new release become a reality. Without the support of people in organisations who believe in the future of QField and open source tool for geospatial in general. The whole team behind QField would like to thank you with a big applause!

What are we looking for?

We would like to use WMS offline on QField. For that, we need to figure out what is the best way to get a raster from a WMS and which format is the most efficient (size and performance).

In this post we’ll show you is how to generate the ideal raster file from a WMS and the results of our efficiency tests for the the different raster formats.

WMS to GPKG

The simple way

If there is no limitation on the WMS or you need only a small region, here is the easiest process.

1. Request the WMS and store a description file in XML :

gdal_translate "WMS:url" file.xml -of WMS

1. Create a Geopackage from the information in the description file.

gdal_translate -of GPKG file.xml file.gpkg -co TILE_FORMAT=JPEG

That was quite simple, right?

The larger datasets way

If the command takes too much time, it means that it is trying to download too much data and could be caused by downloading higher resolution data than required.
The command might even completely fail if it contains a request for bigger data blocks thant the server allows.

Here is the process to get larger datasets in a simple way. Let’s use a real example:

1. Use gdal_translate "WMS:https://www.gebco.net/data_and_products/gebco_web_services/web_map_service/mapserv?request=getmap&service=wms&crs=EPSG:4326&format=image/jpeg&layers=gebco_latest&version=1.1.0" test.xml -of WMS
2. Open the test.xml file for editing, here you’ll find the parameters of the WMS. We change the “SizeX” to 3600 and “SizeY” to 1800. By changing these parameters we lower the resolution. It is important to keep proportionality.
3. Another thing we need to change are “BlockSizeX” and “BlockSizeY” that define the size of the tiles. We change both to 2048.
4. Finally, use gdal_translate -of GPKG test.xml test.gpkg -co TILE_FORMAT=JPEG
5. To make a Geopackage pyramid use gdaladdo GPKG:test.gpkg:gebco_latest. It will replace the Geopackage, if you want to keep the original one, you need to copy it first.

Now you have a raster Geopackage that you can use in QField.

Testing raster formats

Preparing the files

As first step we exported our test orthophoto WMS to a plain GeoTIFF using QGIS’ default behaviour.

Formatgdal_translategdaladdogpkg JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_JPEG.gpkg” -co TILE_FORMAT=JPEG gpkg PNGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG.gpkg” -co TILE_FORMAT=PNGgpkg PNG_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG_JPEG.gpkg” -co TILE_FORMAT=PNG_JPEGgpkg PNG8gdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG8.gpkg” -co TILE_FORMAT=PNG8gpkg WEBPgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_WEBP.gpkg” -co TILE_FORMAT=WEBPgpkg pyramid_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_JPEG.gpkg” -co TILE_FORMAT=JPEGgdaladdo GPKG:C:\test\test_JPEG.gpkg:test_gpkg_JPEG gpkg pyramid_PNGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG.gpkg” -co TILE_FORMAT=PNGgdaladdo GPKG:C:\test\test_PNG.gpkg:test_gpkg_PNGgpkg pyramid_PNG_JPEGgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG_JPEG.gpkg” -co TILE_FORMAT=PNG_JPEGgdaladdo GPKG:C:\test\test_PNG_JPEG.gpkg:test_gpkg_PNG_JPEGgpkg pyramid_PNG8gdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_PNG8.gpkg” -co TILE_FORMAT=PNG8gdaladdo GPKG:C:\test\test_PNG8.gpkg:test_gpkg_PNG8gpkg pyramid_WEBPgdal_translate -of GPKG “C:\test\ortho_test.tif” “C:\test\test_WEBP.gpkg” -co TILE_FORMAT=WEBPgdaladdo GPKG:C:\test\test_WEBP.gpkg:test_gpkg_WEBPJPEG2000gdal_translate -of JP2OpenJPEG “C:\test\ortho_test.tif” “C:\test\test_jpeg_2000.jpg"COG DEFLATEgdal_translate “C:\test\ortho_test.tif” “C:\test\test_cog.tif” -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=DEFLATECOG_JPEGgdal_translate “C:\test\ortho_test.tif” “C:\test\test_cog_JPEG.tif” -co TILED=YES -co COPY_SRC_OVERVIEWS=YES -co COMPRESS=JPEGtifIn QGIS right click on the layer > export > save as > (see the details in the picture under the table)MBTgdal_translate -of MBTILES “C:\test\ortho_test.tif” “C:\test\test_mbt.mbtiles"Creation commands for all the tested formats

Rendering test results

We have tested many formats, here is a table with the results of the size and rendering speed in QGIS and QField.
To analyze the speed we used qgis_bench.exe -i 10 -p "C:\test\test.qgs" >> "C:\test\test.log.
Qgis_bench is a tool that renders a QGIS project a number of times to get performance measurements. The parameter -i is to define the iterations and -p is the project used which contains only the generated raster.

FormatExtent [m]File size [GB]Total_avgTotal_maxdevTotal_minTotal_stdevgpkg JPEG52'880/29'2300.4250.242255.7815.539244.984gpkg PNG52'880/29'2302.9412.002490.328152.142259.859gpkg PNG_JPEG52'880/29'2300.4250.125256.8756.750245.172gpkg PNG852'880/29'2301.4283.875296.40612.625271.250gpkg WEBP52'880/29'2300.3330.238348.10973.534256.703gpkg pyramid_JPEG52'880/29'2300.51.0093.4062.3970.688gpkg pyramid_PNG52'880/29'2303.01.2083.2812.0730.688gpkg pyramid_PNG_JPEG52'880/29'2300.61.4914.3442.8531.016gpkg pyramid_PNG852'880/29'2301.61.5084.3752.8670.969gpkg pyramid_WEBP52'880/29'2300.41.3334.9063.5730.766JPEG200052'880/29'2301.113.888136.109122.2220.219COG DEFLATE52'880/29'2303.6264.427273.09425.411239.016COG_JPEG52'880/29'2301.014.778131.172116.3941.734tif52'880/29'2306.42.3676.7344.3671.672MBT52'880/29'2304.40.4694.6414.1710Comparison of file size and rendering speed of different raster formats. “Total” columns are rendering times in [s]. Lower file size is more storage friendly, lower Total_avg is more performant.

Analysis

File size

The Geopackage WEBP (with and without pyramid) has the best result for file size, but it is not _yet_supported by QField (from 1.6) and is only slightly smaller than the JPEG variant.

Plain GeoTiff, MBTiles, Cloud Optimized GeoTIFF (COG - DEFLATE mode) and Geopackages with PNG generate by far the largest file sizes (up to 20x larger) and are thus not recommended.

Rendering speed

MBTiles are on average double as fast as JPEG Geopackages with pyramids which in turn are more than double as fast as GeoTIFF and 15x faster than COG.
Geopackages without pyramids are 200 to 400 times slower.

Conclusion

Even though MBTiles render faster than the Geopackage pyramid JPEG, they come with an almost 10x bigger storage requirement which makes us say that the best offline raster format supported by QField is Geopackage pyramid JPEG or if you need transparency and slightly smaller files Geopackage pyramid WebP.

If you need transparency before QField 1.6, the best results are achieved with Geopackage pyramid PNG_JPEG.

What a year’s start! After a very packed December publishing all the QGIS on the road videos and quietly releasing QField 1.3 - Ben Nevis we could have gone and relaxed over the holidays. But since we love QField so much we immediately started working on the next iteration. Now, after an intensive testing period, we are proud to announce the release of QField 1.4 - Olavtoppen.

Olavtoppen!? yes, the highest point of Bouvet Island, the remotest island on Earth. And sure enough, QField would follow you there!

As usual, get it on play store or download it from GitHub.

QField Crowdfunding Campaign

Before digging into all the new goodness that you will find in QField 1.4, let’s get a big “Thanks” out to everybody who supported our crowdfunding campaign for improved camera support and all our customers that agreed to open source the work we did for them.

If you like QField, want a new feature or would like to support the project, don’t hesitate to get in touch with us.

Usability enhancements

In QField 1.2 we started to improve on the usability of the user interface. We are constantly working on this with a usability expert to get the user interface to be even more appealing and user-friendly.

Besides lots of clean-up and polishing, QField received two major improvements, a portrait mode and a new welcome screen with recent projects.

Welcome screen with recent projects

QField is all about efficiency. While favourites folders in the file selector already give a great productivity boost, very often we work with the same 3-4 projects. This is why we redesigned the welcome screen to list the last five project used. And if you look carefully you might get a hint of what will be coming soon…

Portrait mode

QField now flawlessly works in portrait mode. We heard you say you needed a comfortable way to work in portrait mode, especially on smartphones. QField forms and button placements are now optimized to be easy to use with your thumbs.

New features

We keep on listening to your feedback and prioritize new features based on it. We did implement some minor features like allowing hiding legend nodes and printing to PDF using the current extent. But this time’s superstars are three highly expected features: Splitting of geometries, compass integration and, yes you guessed right, native camera and gallery app support!

Split Features

A new editing tool is available that allows for splitting existing features. This adds an even more powerful operation to an already impressive geometry editing tools set.

Compass integration

A long-awaited feature! QField now shows you on-screen in which direction you are looking, walking, driving, flying or warping direction. This makes it much easier and more pleasant to navigate in the field.

Native Camera and Gallery

It is now possible to use your favourite camera app so that you have more control over how pictures are taken. It is also possible to select pictures which are already on your device by using the new gallery selector.

Pro Tip: You can use any camera app. For example, you can use the open camera app to create geotagged photos if your preinstalled system camera doesn’t save positioning information in EXIF data.

Pro Tip 2: You can use an image annotation app to add notes, sketches, drawings and so on to your images and then choose them from QField via the add from gallery button.

Antenna Height Correction

For high precision measurements, it’s possible to compensate your altitude by a fixed antenna height. This will then automatically adjust all the digitised altitude values.

JPEG 2000

Support for JPEG 2000 raster datasets was added. This lossy format offers a compression rate at par with proprietary formats like ECW or Mr SID.

Pro Tip: save your base maps in JPEG 2000 to save storage.

New Languages

Thanks to the hard work of our community, QField is now also available in Turkish and Japanese.

New packages

You say: wow that’s a lot! We say: there is more :)
We have upgraded our whole building infrastructure so that you can comfortably get even more QField goodness without having to uninstall your production ready QField.

Automated master builds

After each pull request is merged into our master code, a new package is created and automatically published on the playstore in a dedicated app called QField for QGIS - Unstable (Early Access) . Installing this app will allow you to always have the latest build of QField for testing and giving feedback. On your device, this app is completely separated from the production-ready QField and has a distinctive black icon so that you do not confuse it.

Pull request builds

QField is an extremely active project, and as you see we develop multiple functionalities and fixes at the same time. If you’re particularly interested in one of this, our continuous integration fairy builds and publishes new packages automatically at each commit directly to the pull request you are interested in. To see what we are currently working on, have a look at the pull request overview page .

Experimental Windows builds

Last but definitely not least, we’ve set up an Azure CI infrastructure to build QField for windows. For now, we still consider this experimental but we already had some very successful testing. If you are interested in testing out QField for windows you can get it here , remember it is experimental so don’t use it in production yet and give us as much feedback as possible :)

What’s next?

As you can imagine we’ve had a very busy start of 2020, but even more is to come soon with the next releases of QField. We’d like to thank again all companies and individuals that actively use QField and that invest in making QField even better. If you feel QField misses something you need or would like to support the project, don’t hesitate to get in touch with us.

The goal of the new API is twofold:

1. provide a unified way to store and retrieve data provider connections in the QGIS settings
2. provide an abstract set of methods to perform most common operation on DB data sources (e.g. browse tables, drop/create a table/schema, run arbitrary SQL commands etc.)

The new API is documented in https://qgis.org/api/classQgsAbstractProviderConnection.html and it provides a few specializations for DB connections (https://qgis.org/api/classQgsAbstractDatabaseProviderConnection.html) and an initial PR implementation for web service-based connections (https://github.com/qgis/QGIS/pull/33045).

While the whole of the desired refactoring work was too big for a single grant request, the first work package has been completed and the following data providers have been partially or totally refactored to make use of the new connections API:

• postgres
• geopackage (OGR)
• spatialite

The new API was also used to implement the automatic loading of layer dependencies (not part of the grant program).

For developers interested in working with the new API, a set Python tests are available to show how to use the methods:  https://github.com/qgis/QGIS/blob/master/tests/src/python/test_qgsproviderconnection_ogr_gpkg.py (see also the postgres and spatialite companion tests).

There is still a large amount of work to be done in order to complete all the desired refactoring and to remove all the Python and C++ code that will be ultimately be made redundant. In particular, future work should be undertaken to:

• port all remaining data providers to the new API
• refactor and eliminate the remaining DB-manager connectors to make use of the abstract API
• eliminate duplicate and untested code inside the Processing framework for working with Postgres databases and port the code to the new, stable, well-tested API
• refactor and eliminate the remaining QGIS browser data items to make use of the abstract API 

For further information, the following paragraphs (taken from the original grant proposal) will provide full details about the background of this work.

Background/motivation

• DB-Manager is an important part of the QGIS interface, which allows browsing/previews of different DB-based data sources, complex queries, management of layers, import-export etc., DB creation and deletion etc.
• After the QGIS 3.0 release, improvements within the core browser widgets implemented in C++ have resulted in a (constantly growing) degree of overlapping functionality between the browser and db manager.
• After QGIS 3 API improvements concerning layer import and export functionality, there are many duplicated implementations between browser and db manager – some functionality is better in browser, some functionality is better in db manager. Users are forced to choose between two competing semi-complete alternatives, instead of having one, complete, well integrated solution.
• There are no unit tests for DB-Manager and this leads to frequent regressions, which (aside from being frustrating for users) consume a substantial part of our development time and budget during the bugfixing programs. Furthermore the nature of large Python codebases like db manager makes it very easy to accidentally break functionality with no warning or errors during development.

Proposed solution

We propose to start refactoring the DB-manager plugin functionality into core C++ implementation, reusing existing core API and replacing redundant duplicate functionality.

The clear advantages are:

• no duplicate functionality, so it’s easier for users to understand and use
• more usage of well-tested and well-maintained core C++ API
• testability and immediate feedback on API breaks (an advantage of C++ is that the application wonâ€t even build if an API is changed or accidentally misused)
• better performance
• the ability to expose database management functionality via stable PyQGIS API, allowing other plugins and scripts to utilise this functionality. In future, Processing algorithms may also be developed which would take advantage of these functions (e.g. “create schema”, “drop table”, “vacuum table” algorithms)
• DB management functionality would be available within the main QGIS window (from the Browser panel), instead of as a separate dialog.

Grant proposal package

The above mentioned work is too large to be completed within a single grant, so what we propose here is to start the refactoring needed in order to have a core stable C++ API that can be used by the application and the plugins and that will be available to fully move DB manager to C++ API in the future avoiding duplication of code and functionality.

• create an interface for databases that expose the required functions to a coherent API
• add missing tests and documentation for the a.m. API
• porting some basic functions from db manager to the new api:
  • create table (with native field types support)
  • create schema
  • delete table
  • Rename table

The API will be exposed through the browser and it will be used by the DB manager instead of the Python implementation that is currently used.

After an intensive testing period, we are proud to announce the release of QField 1.2

As usual, get it on play store or download it from GitHub.

QField Crowdfunding Campaign

Before digging into all the new goodness that you will find in QField 1.2, let’s get some big “Thanks” out. What QField currently is was mostly possible thanks to customer projects of which the outcome could be mutualized. Thanks a lot to all of you that agreed open source is all about making things possible together!

Over the years at OPENGIS.ch we have also donated an unimaginable amount of hours to make QField the project you have grown to love and this makes us very proud!

To keep the momentum we now rely on all QField users to help us move one step further. Therefore we created a crowdfunding campaign for improved camera support. As well as another round of general polishing and bug-fixing.

If you like QField, now is the time to show some love and support our crowdfunding campaign .

New features

This new release comes with exciting new features and also contains some first usability enhancements. More of that later.

Value relation widget

If you need to choose the type of a material of the manhole you are inspecting or to select the owner of the parcel which you are drawing, that’s when you want a combo box with available values. This has been possible in QField for a long time, but was hard to set up. Since this release it’s much easier thanks to the integration of value relation widgets.

Not only do they make configuration easier, they also allow for a completely new functionality: managing multiple selections. This will offer a checkbox for every possible value from the list and you are free to save any combination of values.

Authentication dialog for protected services

Just as well as we love open source, we love open data. But not all data are meant for public and some deserve protection. Even more you don’t want to allow everyone to edit your data.

QField will now show an authentication dialog, whenever one of your WMS, WFS, WFS-T or Postgres layers requires a login.

Improved snapping support in expressions

One of the main reasons for QField’s incredible versatility is the use of expressions everywhere. We have just added yet another piece to that: when you snap to a feature, all the snapping details are available for your new feature. With this in place, if you add a new signpost on a street, you can fill in the street_id attribute automatically.

As a nice little extra, the Z (and M) values of snapping results are automatically applied to the new vertices and points.

Usability enhancements and Bugfixes

We also started to improve on the usability of the user interface. We are working on this with a usability expert to get the user interface to be even more appealing and user-friendly.

This is just the start, stay tuned for more usability improvements which are inbound.

As usual, a number of additional bugs have also been corrected, most notably the checkbox widget is now behaving as expected.

Latest Qt 5.13 and Arm64-v8a support

According to Google guidelines, we added support for the Arm64-v8a architecture and while we were at it, we also migrated to the shiny new Qt 5.13 and it’s next-gen menu system.

For this release, we did not upload any x86 packages to the play store since it would have forced us to also have to upload an x86_64 package. If you need the x86 package, you can find it on Github . Obviously, in future releases we’ll add those to the play store as well.

PT | EN

As I was preparing a QGIS Project to read a database structured according to the new rules and technical specifications for the Portuguese Cartography, I started to configure the editing forms for several layers, so that:

1. Make some fields read-only, like for example an identifier field.
2. Configure widgets better suited for each field, to help the user and avoid errors. For example, date-time files with a pop-up calendar, and value lists with dropdown selectors.

Basically, I wanted something like this:

Let me say that, in PostGIS layers, QGIS does a great job in figuring out the best widget to use for each field, as well as the constraints to apply. Which is a great help. Nevertheless, some need some extra configuration.

If I had only a few layers and fields, I would have done them all by hand, but after the 5th layer my personal mantra started to chime in:

“If you are using a computer to perform a repetitive manual task, you are doing it wrong!”

So, I began to think how could I configure the layers and fields more systematically. After some research and trial and error, I came up with the following PyQGIS functions.

Make a field Read-only

The identifier field (“identificador”) is automatically generated by the database. Therefore, the user shouldn’t edit it. So I had better make it read only

To make all the identifier fields read-only, I used the following code.

def field_readonly(layer, fieldname, option = True):
    fields = layer.fields()
    field_idx = fields.indexOf(fieldname)
    if field_idx >= 0:
        form_config = layer.editFormConfig()
        form_config.setReadOnly(field_idx, option)
        layer.setEditFormConfig(form_config)

# Example for the field "identificador"

project = QgsProject.instance()
layers = project.mapLayers() 

for layer in layers.values():
    field_readonly(layer,'identificador')

Set fields with DateTime widget

The date fields are configured automatically, but the default widget setting only outputs the date, and not date-time, as the rules required.

I started by setting a field in a layer exactly how I wanted, then I tried to figure out how those setting were saved in PyQGIS using the Python console:

>>>layer = iface.mapCanvas().currentLayer()
>>>layer.fields().indexOf('inicio_objeto')
1
>>>field = layer.fields()[1]
>>>field.editorWidgetSetup().type()
'DateTime'
>>>field.editorWidgetSetup().config()
{'allow_null': True, 'calendar_popup': True, 'display_format': 'yyyy-MM-dd HH:mm:ss', 'field_format': 'yyyy-MM-dd HH:mm:ss', 'field_iso_format': False}

Knowing this, I was able to create a function that allows configuring a field in a layer using the exact same settings, and apply it to all layers.

def field_to_datetime(layer, fieldname):
    config = {'allow_null': True,
              'calendar_popup': True,
              'display_format': 'yyyy-MM-dd HH:mm:ss',
              'field_format': 'yyyy-MM-dd HH:mm:ss',
              'field_iso_format': False}
    type = 'Datetime'
    fields = layer.fields()
    field_idx = fields.indexOf(fieldname)
    if field_idx >= 0:
        widget_setup = QgsEditorWidgetSetup(type,config)
        layer.setEditorWidgetSetup(field_idx, widget_setup)

# Example applied to "inicio_objeto" e "fim_objeto"

for layer in layers.values():
    field_to_datetime(layer,'inicio_objeto')
    field_to_datetime(layer,'fim_objeto')

Setting a field with the Value Relation widget

In the data model, many tables have fields that only allow a limited number of values. Those values are referenced to other tables, the Foreign keys.

In these cases, it’s quite helpful to use a Value Relation widget. To configure fields with it in a programmatic way, it’s quite similar to the earlier example, where we first neet to set an example and see how it’s stored, but in this case, each field has a slightly different settings

Luckily, whoever designed the data model, did a favor to us all by giving the same name to the fields and the related tables, making it possible to automatically adapt the settings for each case.

The function stars by gathering all fields in which the name starts with ‘valor_’ (value). Then, iterating over those fields, adapts the configuration to use the reference layer that as the same name as the field.

def field_to_value_relation(layer):
    fields = layer.fields()
    pattern = re.compile(r'^valor_')
    fields_valor = [field for field in fields if pattern.match(field.name())]
    if len(fields_valor) > 0:
        config = {'AllowMulti': False,
                  'AllowNull': True,
                  'FilterExpression': '',
                  'Key': 'identificador',
                  'Layer': '',
                  'NofColumns': 1,
                  'OrderByValue': False,
                  'UseCompleter': False,
                   'Value': 'descricao'}
        for field in fields_valor:
            field_idx = fields.indexOf(field.name())
            if field_idx >= 0:
                print(field)
                try:
                    target_layer = QgsProject.instance().mapLayersByName(field.name())[0]
                    config['Layer'] = target_layer.id()
                    widget_setup = QgsEditorWidgetSetup('ValueRelation',config)
                    layer.setEditorWidgetSetup(field_idx, widget_setup)
                except:
                    pass
            else:
                return False
    else:
        return False
    return True
    
# Correr função em todas as camadas
for layer in layers.values():
    field_to_value_relation(layer)

Conclusion

In a relatively quick way, I was able to set all the project’s layers with the widgets I needed.

This seems to me like the tip of the iceberg. If one has the need, with some search and patience, other configurations can be changed using PyQGIS. Therefore, think twice before embarking in configuring a big project, layer by layer, field by fields.

QGIS recipes have been available on Conda for a while, but now, that they work for the three main operating systems, getting QGIS from Conda is s starting to become a reliable alternative to other QGIS distributions. Anyway, let’s rewind a bit…

What is Conda?

Conda is an open source package management system and environment management system that runs on Windows, macOS and Linux. Conda quickly installs, runs and updates packages and their dependencies. Conda easily creates, saves, loads and switches between environments on your local computer. It was created for Python programs, but it can package and distribute software for any language.

Why is that of any relevance?

Conda provides a similar way to build, package and install QGIS (or any other software) in Linux, Windows, and Mac.

As a user, it’s the installation part that I enjoy the most. I am a Linux user, and one of the significant limitations is not having an easy way to install more than one version of QGIS on my machine (for example the latest stable version and the Long Term Release). I was able to work around that limitation by compiling QGIS myself, but with Conda, I can install as many versions as I want in a very convenient way.

The following paragraphs explain how to install QGIS using Conda. The instructions and Conda commands should be quite similar for all the operating systems.

Anaconda or miniconda?

First thing you need to do is to install the Conda packaging system. Two distributions install Conda: Anaconda and Miniconda.

TL;DR Anaconda is big (3Gb?) and installs the packaging system and a lot of useful tools, python packages, libraries, etc… . Miniconda is much smaller and installs just the packaging system, which is the bare minimum that you need to work with Conda and will allow you to selectively install the tools and packages you need. I prefer the later.

For more information, check this stack exchange answer on anaconda vs miniconda.

Download anaconda or miniconda installers for your system and follow the instructions to install it.

Windows installer is an executable, you should run it as administrator. The OSX and Linux installers are bash scripts, which means that, once downloaded, you need to run something like this to install:

bash Miniconda3-latest-Linux-x86_64.sh

Installing QGIS

Notice that the Conda tools are used in a command line terminal. Besides, on Windows, you need to use the command prompt that is installed with miniconda.

Using environments

Conda works with environments, which are similar to Python virtual environments but not limited only to python. Basically, it allows isolating different installations or setups without interfering with the rest of the system. I recommend that you always use environments. If, like me, you want to have more that one version of QGIS installed, then the use of environments is mandatory.

Creating an environment is as easy as entering the following command on the terminal:

conda create --name <name_of_the_environment>

For example,

conda create --name qgis_stable

You can choose the version of python to use in your environment by adding the option python=<version>. Currently versions of QGIS run on python 3.6, 3.7, 3.8 and 3.9.

conda create –name qgis_stable python=3.7

To use an environment, you need to activate it.

conda activate qgis_stable

Your terminal prompt will show you the active environment.

(qgis_stable) aneto@oryx:~/miniconda3$

To deactivate the current environment, you run

conda deactivate

Installing packages

Installing packages using Conda is as simples as:

conda install <package_name>

Because conda packages can be stored in different channels, and because the default channels (from the anaconda service) do not contain QGIS, we need to specify the channel we want to get the package from. conda-forge is a community-driven repository of conda recipes and includes updated QGIS packages.

conda install qgis --channel conda-forge

Conda will download the latest available version of QGIS and all its dependencies installing it on the active environment.

Note: Because conda always try to install the latest version, if you want to use the QGIS LTR version, you must specify the QGIS version.

conda install qgis=3.10.12 --channel conda-forge

Uninstalling packages

Uninstalling QGIS is also easy. The quickest option is to delete the entire environment where QGIS was installed. Make sure you deactivate it first.

conda deactivate
conda env remove --name qgis_stable

Another option is to remove QGIS package manually. This is useful if you have other packages installed that you want to keep.

conda activate qgis_stable
conda remove qgis -c conda-forge

This only removes the QGIS package and will leave all other packages that were installed with it. Note that you need to specify the conda-forge channel. Otherwise, Conda will try to update some packages from the default channels during the removal process, and things may get messy.

Running QGIS

To run QGIS, in the terminal, activate the environment (if not activated already) and run the qgis command

conda activate qgis_stable
qgis

Updating QGIS

To update QGIS to the most recent version, you need to run the following command with the respective environment active

conda update qgis -c conda-forge

To update a patch release for the QGIS LTR version you run the install command again with the new version:

conda install qgis=3.10.13 -c conda-forge

Some notes and caveats

Please be aware that QGIS packages on Conda do not provide the same level of user experience as the official Linux, Windows, and Mac installer from the QGIS.org distribution. For example, there are no desktop icons or file association, it does not include GRASS and SAGA, etc …

On the other hand, QGIS installations on Conda it will share user configurations, installed plugins, with any other QGIS installations on your system.