Page 1 of 1 (9 posts)

  • talks about »
  • ubuntu

Tags

Last update:
Wed Aug 27 07:50:07 2014

A Django site.

QGIS Planet

Building QGIS on Ubuntu 13.04

It seems upgrading to Ubuntu 13.04 breaks a few things in the QGIS build process. Here’s some quick workarounds I’ve found to get it building again.

Firstly, Ubuntu defaults to Qt 5, causing the error:

CMake Error at /usr/share/cmake-2.8/Modules/FindQt4.cmake:1216 (message):

 Found unsuitable Qt version "5.0.1" from /usr/bin/qmake, this code requires
 Qt 4.x

This can be fixed by switching the system default back to Qt 4, so that  qmake refers to the Qt 4 version. The “qt4-default” package handles this switch for you, so just run:

sudo apt-get install qt4-default

Update: A better solution was found by Pierre and Pvanb and is described here. Basically it involves changing the QMAKE_EXECUTABLE option from /usr/bin/qmake to /usr/bin/qmake-qt4.

Next, I was getting the error:

make[2]: *** No rule to make target `/usr/lib/libpython2.7.so', needed by `output/lib/libqgispython.so.1.9.0'. Stop.

I’m not sure if this is the correct solution, but setting up some links and re-running ldconfig gets around this:

sudo ln -sf /usr/lib/x86_64-linux-gnu/libpython2.7.so.1 /usr/lib/libpython2.7.so.1
sudo ln -sf /usr/lib/x86_64-linux-gnu/libpython2.7.so /usr/lib/libpython2.7.so
sudo ldconfig

Update: a better solution is to change the PYTHON_LIBRARY option from /usr/lib/libpython2.7.so to /usr/lib/x86_64-linux-gnu/libpython2.7.so

These two changes were enough to get QGIS building again. If you’ve got a better solution for these errors, let me know…

Automatically restarting services after upgrades on Debian and Ubuntu

There are various tools to automatically keep a Debian/Ubuntu system security wise up to date, among others the unattended-upgrades package.

Also, there’s the checkrestart script from the debian-goodies package, that scans all the open files on a system and tries to determine to what service they belong and how that service might be restarted.

The last piece that’d tie all those scripts together and would automatically restart all services that are using stale libraries or files was missing.

With the help of Michal Fiala there however is now the restart-services script, that does just that.

The script has not seen much real world usage and as such should be regarded as experimental (f.ex. by restarting /etc/init.d/screen it will as of the time of writing terminate existing screen sessions).

The script currently lives on Github. If you encounter any problem with the script then we’ll very much wellcome a patch that fixes it…

Tomáš Pospíšek

Update 18.6.2012: The most recent checkrestart (from debian-goodies 0.61) now excludes screen from beeing listed among the services to be restarted.

Automatically restarting services after upgrades on Debian and Ubuntu

There are various tools to automatically keep a Debian/Ubuntu system security wise up to date, among others the unattended-upgrades package.

Also, there’s the checkrestart script from the debian-goodies package, that scans all the open files on a system and tries to determine to what service they belong and how that service might be restarted.

The last piece that’d tie all those scripts together and would automatically restart all services that are using stale libraries or files was missing.

With the help of Michal Fiala there however is now the restart-services script, that does just that.

The script has not seen much real world usage and as such should be regarded as experimental (f.ex. by restarting /etc/init.d/screen it will as of the time of writing terminate existing screen sessions).

The script currently lives on Github. If you encounter any problem with the script then we’ll very much wellcome a patch that fixes it…

Tomáš Pospíšek

Update 18.6.2012: The most recent checkrestart (from debian-goodies 0.61) now excludes screen from beeing listed among the services to be restarted.

USB Recovery Script

What do you do when you are managing a remote server and you need to make some critical changes (like to the networking configs) and you feel uncomfortable about the possibility of losing access to the server and never getting it back? This was the situation we were in today. The server is a little esoteric - its a headless box and even in the server center the engineers don't have any way to log in interactively at the server itself. Luckily the server is running Debian linux and has a usb port so help is at hand via bash!

I wrote this little script which is designed to be run from a cron job, for example every minute.

#!/bin/bash

# This script is to rescue the system from usb while
# testing migration to the new vpn.

# It will mount the last partition of any inserted usb,
# cd to the mount point and try to run a script
# called 'rescue.sh'
# After the script is run it will be renamed to
# rescue.ok
#
# You should set this script to run as a cron job
# at minute intervals.
#
# e.g. # m h  dom mon dow   command
#      * * * * * /root/usbrescue.sh
#

RESCUEFILE=rescue.sh
OKFILE=rescue.ok
LOGFILE=rescue.log
MOUNTPOINT=/mnt/rescue
SCRIPTPATH=${MOUNTPOINT}/${RESCUEFILE}
OKPATH=${MOUNTPOINT}/${OKFILE}
LOGPATH=${MOUNTPOINT}/${LOGFILE}
# Note we ignore partitions on devices sda - sdd as they are internal disks
LASTPARTITION=$(cat /proc/partitions  | awk '{print $4}' | grep -v 'sd[a-d]' \
| grep -v name | grep -v '^$' |sort | tail -1)
if [ $LASTPARTITION != "" ]
then
  if [ ! -b /dev/$LASTPARTITION ]
  then
    echo "Error /dev/$LASTPARTITION is not a block device"
    exit
  else
    echo "OK /dev/$LASTPARTITION is a block device"
  fi
  echo "Device found creating mount point"
  if [ ! -d "$MOUNTPOINT" ]
  then
    mkdir $MOUNTPOINT
  fi
  echo "Mounting...."
  mount /dev/$LASTPARTITION $MOUNTPOINT
  echo "Checking if rescue script exists"
  # Test the rescue script exists(-e) and is not 0 length (-s)
  if [ -e $SCRIPTPATH -a -s $SCRIPTPATH ]
  then
    echo "Making $SCRIPTPATH executable"
    chmod +x $SCRIPTPATH
    echo "Running script"
    $SCRIPTPATH > $LOGPATH 2>&1
    echo "Disabling script"
    mv $SCRIPTPATH $OKPATH
  else
    echo "No Rescue script found"
  fi
  echo "Unmounting.."
  cd /
  umount $MOUNTPOINT
else
  echo "No rescue device found"
fi
echo "done"

If you place the script in /root/usbrescue.sh and add a cron job as outlined in the comments, it will poll for devices regularly, mount the last partition available.

If it finds a script on that partition labelled rescue.sh, it will run it then rename the script to rescue.ok and write any stderror and stdout logs to rescue.log on the partition. The script could perhaps be improved by adding a lock file so that it does not get run again if it is already running (if it takes longer than a minute to run for example), but it's a good starting point for a system rescue if things go wrong. Now the engineer on site can simply pop in his usb stick and any recovery commands will be run from it.

Ubuntu PostGIS package for PostgreSQL 9.0

A while ago I’ve compiled PostGIS 1.5.2 for PostgreSQL 9.0.

To install it on Ubuntu the following steps are required:

apt-get install python-software-properties

add-apt-repository ppa:ubuntugis/ubuntugis-unstable
add-apt-repository ppa:pitti/postgresql
add-apt-repository ppa:pi-deb/gis

apt-get update

apt-get install postgresql-9.0-postgis

A basic template database can be created with the following commands:

sudo su - postgres

createdb template_postgis
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/postgis.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/spatial_ref_sys.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis_comments.sql
cat <<EOS | psql -d template_postgis
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
REVOKE ALL ON SCHEMA public FROM public;
GRANT USAGE ON SCHEMA public TO public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT SELECT, UPDATE, INSERT, DELETE
  ON TABLE public.geometry_columns TO PUBLIC;
GRANT SELECT, UPDATE, INSERT, DELETE
  ON TABLE public.spatial_ref_sys TO PUBLIC;
EOS

To test database creation you can do the following:

createdb --template template_postgis test_gis
psql -d test_gis -c "select postgis_lib_version();"

Ubuntu supports parallel installations of different PostgreSQL versions. So if you have already PostgreSQL 8.x installed, PostgreSQL 9.0 will probably be configured to listen on port 5433. So you have to add the option -p 5433 to each command and to specify the full path for the executables. For example:

/usr/lib/postgresql/9.0/bin/psql -p 5433 -d test_gis -c "select postgis_lib_version();"

Ubuntu PostGIS package for PostgreSQL 9.0

A while ago I’ve compiled PostGIS 1.5.2 for PostgreSQL 9.0.

To install it on Ubuntu the following steps are required:

apt-get install python-software-properties

add-apt-repository ppa:ubuntugis/ubuntugis-unstable
add-apt-repository ppa:pitti/postgresql
add-apt-repository ppa:pi-deb/gis

apt-get update

apt-get install postgresql-9.0-postgis

A basic template database can be created with the following commands:

sudo su - postgres

createdb template_postgis
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/postgis.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis-1.5/spatial_ref_sys.sql
psql -q -d template_postgis -f /usr/share/postgresql/9.0/contrib/postgis_comments.sql
cat <<EOS | psql -d template_postgis
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
REVOKE ALL ON SCHEMA public FROM public;
GRANT USAGE ON SCHEMA public TO public;
GRANT ALL ON SCHEMA public TO postgres;
GRANT SELECT, UPDATE, INSERT, DELETE
  ON TABLE public.geometry_columns TO PUBLIC;
GRANT SELECT, UPDATE, INSERT, DELETE
  ON TABLE public.spatial_ref_sys TO PUBLIC;
EOS

To test database creation you can do the following:

createdb --template template_postgis test_gis
psql -d test_gis -c "select postgis_lib_version();"

Ubuntu supports parallel installations of different PostgreSQL versions. So if you have already PostgreSQL 8.x installed, PostgreSQL 9.0 will probably be configured to listen on port 5433. So you have to add the option -p 5433 to each command and to specify the full path for the executables. For example:

/usr/lib/postgresql/9.0/bin/psql -p 5433 -d test_gis -c "select postgis_lib_version();"

Creating and applying patches with Git

During the recent QGIS hackfest in Poland, we spent some time discussing the use of GIT and I spent a bit of time to learn the basics of using GIT. One thing about GIT that is different for users like myself coming from an SVN background is the way that creating and applying patches is done. Typically under svn I would do something like this to create a patch:

svn diff > foo.diff

And then to apply that same patch I would do:

patch -p0 < foo.diff

Which is all quite simple. Git's process is a little different. Here is my workflow for creating a patch:

  • Create a branch (GIT seems to encourage you to work in branches and to branch often)
  • Checkout your branch
  • Work and change things in your branch
  • Commit your work to your branch
  • Generate a patch as a diff between your branch and master
  • Submit your patch
  • Apply your patch

So here is a simple session that does the above:

git branch patch-testing
git checkout patch-testing
( do some work in that branch now)
git commit -m "Important changes" -a
git format-patch master

After you are done with that,  Git will have created a nice little diff for you. To apply the patch to another checkout do:

git apply foo.path

Then commit and push your changes to the origin repository.

A workflow for creating beautiful relief shaded dems using GDAL

Sometimes I create hillshades using the QGIS hillshade plugin and then overlay the original DEM over it. I set the DEM to have a false colour pallette and set it to be semi-transparent to produce something like this:

Typical usage of a hillshade with false colour overlay

That is all well and good but a bit impractical. It would be much nice to have the colour pallette permanetly assigned to the hillshade. Also I want to be able to clip and mask the resulting raster to the boundary specified in a shapefile. Fortunately, GDAL provides all the tools you need to make this happen. There are already some nice articles (like this one) that describe parts of this workflow but I am writing this because I wanted to note the additional steps that I took to make this work well for me.

Before you begin

Before you begin you should have:

  1. a raster containing digital elevation data (DEM) - in my case its called 'alt.tif'
  2. a vector layer (typically a shapefile) containing the area of interest for your final product - in my case its called 'tanzania.shp'

Create the hillshade image

The first thing we need to do is generate a hillshade. There is a nice python plugin for doing this in QGIS, you can do it in GRASS using the QGIS-GRASS plugin. But in this article I'm going for an all-GDAL approach so we will be using the handy **gdaldem** command line application.  I won't be explaining the parameters I have used here as they are well explained in the gdaldem manual pages.

So to create our hillshade we do something like this:

::
gdaldem hillshade alt.tif shade.tif -z 5 -s 111120 -az 90

Which will produce something like this: colorbrewer would be a good place to start if you want to learn more.  Another favourite site of mine is colourlovers.com and for this tutorial I decided to use a pallette from there to see how it would turn out.

Once you have selected a suitable colour pallette (around 5 or 6 colour classes should do it), the next thing you need to do is get some information about the range of values contained in your DEM. Once again you can easily point and click your way to this in QGIS, but here is the way to get it in gdal from the command line:

::
gdalinfo -mm alt.tif

The resulting output includes the computed min/max for Band 1 - which is what we are after:

::
Band 1 Block=1334x3 Type=UInt16, ColorInterp=Gray
Computed Min/Max=1.000,5768.000 NoData Value=65535

Ok so our minimum height is 1m and maximum is 5768m - Tanzania is the home of Kilimanjaro after all! So lets split that range up into 5 classes to match the 'Landcarpet Europe' colourlover pallette I selected. I set nearly white as an additional colour for the highest altitude range.

::
65535 255 255 255 5800 254 254 254 3000 121 117 10 1500 151 106 47 800 127 166 122 500 213 213 149 1 218 179 122

The value in the first column is the base of the scale range. The subsequent values are RGB triplets for that range. I saved this as a text file called 'ramp.txt' in the same directory as my 'alt.tiff' dataset. You will notice that I made the value ranges closer together at lower altitudes and wider appart at higher altitudes. You may want to experiment a little to get pleasing results - especially if you have a relatively small number of high lying terrain pixels and the rest confined to lower lying areas.

Also note that I assigned the highest terrain 'nearly white' so that I could reserve white (RGB: 255 255 255) for the nodata value (65535) in this dataset. We will use the fact that white is only used for nodata to our advantage when we do a clip further on in these instructions.

Ok now we can use gdaldem to generate the terrain map:

::
gdaldem color-relief alt.tif ramp.txt relief.tif

This is what my relief map looked like:

The terrain colour map I produced

Don't worry about the fact that it does not resemble the colour pallette you have chosen - it will do in the next step!

Merging the two rasters

The next step is to combine the two products. I used Frank Warmerdam's handy hsv_merge.py script for this purpose.

::
./hsv_merge.py relief.tif shade.tif colour_shade.tif

Which produced this:

The result of merging my hillshade and my terrain colour map

You may have noticed that it is only at this point that the colours of our product resemble the original pallette we used.

One little gotcha with the hsv_merge.py script is that it throws away our nodata values, so what was sea before (and nodata in my original alt.tif dataset) is now white (RGB: 255 255 255).

Clipping and masking

You may have everything you need from the above steps. Alternatively you can also clip and mask the dataset using a shapefile.

::
gdalwarp -co compress=deflate -dstnodata 255 -cutline Tanzania.shp
colour_shade.tif colour_shade_clipped.tif

My final image is now a compressed tif with nodata outside of the country of Tanzania and looks like this:

Final result: A false coloured elevation map for Tanzania

A final note

One of the things I love about the command line is the repeatable and reusable workflows it allows for. I can distill everything in this blog post into a sequence of commands and replay them any time. If you are still stuck doing things in a GUI only way, give BASH a try - you can really do powerful geospatial data processing with it!

Ubuntu Jaunty Post-Installation for FOSSGIS

It's quick (overlooking slow internet access) and easy to set up Ubuntu Jaunty 64 bit for use with the FOSSGIS stack. Here are the packages I install once the base system is set up (including a few non-fossgis ones that are nice to have).

sudo apt-get install build-essential vim grass libqt4-core \
libqt4-debug libqt4-dev libqt4-gui libqt4-qt3support libqt4-sql \
lsb-qt4 qt4-designer   qt4-dev-tools qt4-doc qt4-qtconfig uim-qt gcc \
libapt-pkg-perl resolvconf   gdal-bin libgdal1-dev libgeos-dev proj \
libgdal-doc libhdf4g-dev libhdf4g-run python-dev  python-gdal libgsl0-dev \
g++ libjasper-dev libtiff4-dev subversion libsqlite3-dev sqlite3 ccache \
make libpq-dev flex bison cmake txt2tags python-qt4 python-qt4-dev \
python-sip4 sip4 python-sip4-dev postgresql-8.3-postgis vim ctags \
vim-scripts python-django python-psycopg2 vim apache2 \
sun-java6-fonts python-geoip imagemagick grass-dev ia32-libs \
graphviz txt2tags texlive-fonts-recommended \
texlive-fonts-recommended-doc texlive-latex-base texlive-latex-base-doc \
tipa tex-common texlive-base texlive-base-bin texlive-base-bin-doc \
texlive-common texlive-doc-base gv latex2html texlive-latex-extra \
libqwt5-qt4-dev pyqt4-dev-tools drivel lightning-extension thunderbird-gnome-support \
gwibber python-django-evolution python-django-dmigrations python-django-djblets \
python-django-debug-toolbar python-django-registration python-django-lint \
python-django-tagging awn-applets-c-extras awn-applets-python-extras\
gnome-do tomcat6-admin tomcat6 enigmail

You also need to get rid of this since it doesnt play nice with QGIS:

sudo apt-get remove uim-qt3

Note: the ia32-libs is needed to run google earth under ubuntu 64

  • Page 1 of 1 ( 9 posts )
  • ubuntu

Back to Top

Sponsors