PerryGeo

In the why-haven’t-I-ever-heard-of-this department:

Mirone is a Windows MATLAB-based framework tool that allows the display and manipulation of a large number of grid formats through its interface with the GDAL library. Its main purpose is to provide users with an easy-to-use graphical interface to the more commonly used programs of the GMT package.

There is also a version that does not depend on MATLAB which is what I decided to try. This is a great package; easy to install, very usable, lots of high-end raster functionality, and a good sense of humor…

Considering GMT and GDAL can be a bit challenging and unfamiliar for a typical windows user, Mirone is a huge step forward.

Among some of the functionality that is an absolute pleasure to work with compared to some other software packages: surface profiles, image-flipping, DEM derivatives, color-ramping, contouring, histograms, kernel filtering… And that’s just scratching the surface. I highly recommend checking it out.

It just keeps getting easier and easier to get a fully functional open source GIS workstation up and running thanks to Ubuntu. The following instructions will take your vanilla installation of Ubuntu 7.10 and add the following top-notch desktop GIS applications:

• Postgresql/PostGIS : a relational database with vector spatial data handling
• GRASS : A full blown GIS analysis toolset
• Quantum GIS: A user-friendly graphical GIS application
• GDAL, Proj, Geos : Libraries and utilities for processing spatial data
• Mapserver : web mapping program and utilites
• Python bindings for QGIS, mapserver and GDAL
• GPSBabel : for converting between various GPS formats
• R : a high-end statistics package with spatial capabilities
• GMT : the Generic Mapping Tools for automated high-quality map output

While this is not a comprehensive list of open source GIS software, these packages cover most of my needs. If you want to live on the bleeding edge and have to have the absolute latest versions, you’ll be better off installing these from source. But for those of us that want a stable and highly functional GIS workstation with minimal fuss, this is the way to go:

1. Go to System > Administration > Software Sources and make sure the universe and multiverse repositories are turned on. Close the window and the list of available software packages will be refreshed.
2. Open up a terminal (ie the command line) via Applications > Accessories > Terminal and type the following:
   

   sudo apt-get -y install qgis grass qgis-plugin-grass mapserver-bin gdal-bin cgi-mapserver \
   python-qt4 python-sip4 python-gdal python-mapscript gmt gmt-coastline-data \
   r-recommended gpsbabel shapelib libgdal1-1.4.0-grass

   The sudo part indicates that the command will be run as the administrator user, apt-get -y install is the command telling it to install the list of packages and answer yes to any questions that pop up.

3. There is one package that is worth upgrading to the latest and greatest - Quantum GIS. The latest version (0.9) is due out very shortly and has the ability to write plugins using the python programming language. A big plus!

   Download the latest build from http://qgis.org/uploadfiles/testbuilds/qgis0.9.0.debs_ubuntu_gutsy.tar.gz and extract it ( right-click > Extract Here ). In the directory you’ll see 4 .deb files, only 3 of which you’ll need unless you plan on doing any development work.

   Double click libqgis1_0.9.0_i386.deb and you’ll get a message saying an older version is available from directly from ubuntu. We already know this so just close and ignore it. Click Install Package and wait for it to complete then close out.

   Repeat for qgis_0.9.0_i386.deb and qgis-plugin-grass_0.9.0_i386.deb (in that order).

And there we have it, about 15 minutes depending on your internet speed and you’ve installed a high-end GIS workstation built completely on free and open source software.

Not very often do I see open source mentioned on the front page of my Google Finance page (let alone Geospatial Open Source). But here it is.. the announcement was made at FOSS4G2007 that autodesk will be open sourcing part of it’s coordinate system and map projection technology.

So what motivation does Autodesk (or any other company) have to open source it’s technology? An important line from Lisa Campbell, vice president, Autodesk Geospatial:

“Our intent to contribute again to the open source community is a reflection of our customers’ desire for faster innovation, more frequent product releases, and lower total cost of ownership.”

FDO, GDAL and FME all seem to operate in roughly the same domain - Providing a data model, API and tools to translate between spatial data formats. Does anyone know of any good write-ups comparing/contrasting the features of these three libraries?

A few weeks back, I decided to take the plunge and learn the python bindings for QGIS 0.9. My first experiment was to implement a geocoder plugin. What started mostly as a learning experiment turned into something that might actually be useful!

The idea was to use web services to do all the actual geocoding work (the hard part!) and the delimited text provider to load the results into qgis. Right now it’s built on top of the Yahoo geocoder which is, IMO, the best out there.. very flexible about the input format. The geopy module is used to interact with the geocoding services so it could potentially support other engines such as geocoder.us, virtual earth, google, etc.

The user interface is very straightforward; enter list of addresses/placenames seperated by a line break, pick an output file and go. To be legitimate, you should also sign up for a yahoo api key, though the ‘YahooDemo’ key will work ok for testing purposes.

Here’s the install process (assuming you already have python, pyqt4, qgis 0.9, qgis bindings, etc. set up):

 svn checkout http://perrygeo.googlecode.com/svn/trunk/qgis/geocode
 cd geocode
 emacs Makefile # change install directory if needed
 sudo make install

This is just a rough cut and it’s my first attempt at using the qgis and qt apis so there are probably many things that could be improved upon. Ideally this plugin could:

• Parse text files as input
• Allow for a choice of geocoding engine
• ???

Feedback (and patches) welcome

Though probably not as big of a news item as this week’s release of Ubuntu Dapper, there’s another Linux release that might be of interest to us GIS folk:

Built off of a Slackware base (one of the oldest, most stable linux distros), HostGIS Linux aims to be a “minimal yet complete” distribution specifically built with GIS in mind. It is first and foremost a server platform; it does not include any window system at all. If you’re looking for desktop GIS applications out-of-box, it might not be the best for you.

But for a GIS server, it comes with most of the open source stack preinstalled and configured. This latest release has a few changes and version upgrades for most of the components.

• PHP, Python and Perl Mapscript
• GDAL/OGR with PHP, Python and perl bindings
• Postgresql 8.1 with PostGIS 1.1
• drivers for many extra formats including jpeg2000 and ecw
• Apache web server with Mapserver CGI

The primary motivation for creating HGL was to speed up the installation of new gis-enabled servers. Gregor Mosheh, the head programmer for HostGIS, has done an excellent job pretty much single-handedly putting this together. ( In full disclosure, I do consulting work for HostGIS, though I wasn’t really involved in the creation of HostGIS Linux. )

The setup is your standard text-based install and is a piece of cake if you’ve ever installed Linux before. When you’re through, you have the good ole’ black and white text console staring at you. Not very interesting… But the really satisfying part is to fire up a web browser after the install and be able to point it to a working webGIS application. Anyone who has spent the time to set up the mapserver stack and its seemingly infinite dependencies can appreciate the amount of work this saves!

If you’re not into learning a new distro, there is always the FGS linux installer which will set up a similar software stack on pretty much any linux.

And for Desktop GIS, many linux distros have a selection of GIS apps in their package repositories (You’ll want to certainly grab GRASS, GDAL and QGIS) . FWTools can be a good option on both Linux and Windows to get you up and running quickly. Finally there are a number of other more desktop-oriented distros for GIS including Knoppix GIS and GeoLivre, both of which run as a live-cd so you can check it out before you install.

Anyways, back to sum up HostGIS Linux:

If you need to set up a GIS server with minimal fuss and you have some experience with Linux, you might like to try it out. It will save lots of time.

If you’re a GIS user who needs a graphic windows environment to do GIS work on the Desktop, HostGIS Linux will not really make you happy out-of-the-box. Of course, since HGL is slackware based, you can use the slackware package management system to build an impressive Desktop system. But if you don’t need to run a server or really care about having the latest versions, Ubuntu comes with a solid desktop environment and packages for alot of good GIS apps.

A few months ago, I began looking for some efficient command-line tools to analyze and visualize DEMs. I typically use GRASS for such tasks but GRASS only works with it’s native raster format. Sure you can import/export to common formats but that’s not as efficient as a single command line tool that could work with the native DEM format, run on systems without GRASS installed and provide easy scriptablity.

Not having found anything that fit the bill, I decided to port some of the common GRASS DEM modules to C++ using the GDAL libraries. For someone with very little experience with C++, this was surprisingly not that difficult though I learned quite alot along the way. The result: 3 command line utilities to generate hillshades, slope and aspect maps and 1 excellent utility contributed by Paul Surgeon to apply color ramping to a DEM.

Installation

Requirements

I built these utilities on Ubuntu Linux. I admittedly have no idea how to compile them on Windows but some folks have confirmed that the hillshade code compiles under VC++. So to get these running under Linux (and presumably other *nixes), there are very minimal requirements:

1. GDAL shared libraries
2. GNU C++ Compiler

Download

Get the current source and unzip it. EDIT : This code is now avaible through my SVN repository : http://perrygeo.googlecode.com/svn/trunk/demtools/,

Compiling

Alas there is no makefile but installation should be fairly painless. To compile the source code under linux, the following commands should take care of it:

g++ hillshade.cpp -lgdal -o hillshade
g++ color-relief.cxx -lgdal -o color-relief
g++ aspect.cpp -lgdal -o aspect
g++ slope.cpp -lgdal -o slope

The four binaries can then be placed wherever your local binaries reside (typically /usr/local/bin)

Examples

The original DEM

In this particular example the input DEM is a GeoTIFF but these utilities can use any GDAL-supported raster source.

Slope

This command will take a DEM raster and output a 32-bit GeoTiff with slope values. You have the option of specifying the type of slope value you want: degrees or percent slope. In cases where the horizontal units differ from the vertical units, you can also supply a scaling factor.

slope dem.tif slope.tif

Aspect

This command outputs a 32-bit GeoTiff with values between 0 and 360 representing the azimuth of the terrain.

aspect dem.tif aspect.tif

Hillshade

This command outputs an 8-bit GeoTiff with a nice shaded relief effect. It’s very useful for visualizing the terrain. You can optionally specify the azimuth and altitude of the light source, a vertical exaggeration factor and a scaling factor to account for differences between vertical and horizontal units.

hillshade dem.tif shade.tif

Color ramps

After I posted the hillshade utility to the gdal-dev mailing list, there was some discussion about creating color relief maps to supplement the hillshades. Paul Surgeon took up the challenge and created a gdal-based C++ utility to colorize DEMs (or any other single band raster data sources for that matter). The technique is simple and powerful; by using a text-based color configuration file, you can create any range of color ramps for your data.

color-relief dem.tif scale.txt colordem.tif

Where scale.txt is a text file containting 4 columns per line, the elevation value and the corresponding RGB values:

3500   255 255 255
2500   235 220 175
1500   190 185 135
700    240 250 150
0      50  180  50
-32768 200 230 255

The colors between the given elevation values are blended smoothly and the result is a nice colorized DEM:

Color Shaded Relief (blending hillshade and colorized DEM)

There are two ways I’ve come up with to blend the hillshade and the colorized DEM:

1. Using GIMP or Photoshop, open both images, copy the shaded relief, paste on top of the color DEM and adjust the opacity in the layers dialog.
2. If you’re publishing to the web with Mapserver, just stack the two images in your mapfile and set the TRANSPARENCY for the hillshade to a value between 30 and 70 depending on your preference

Though both methods work nicely, neither is really ideal since they don’t generate a georeferenced tiff. You can get around this in the GIMP method by creating a world file (.tfw) for the output tiff. It might be nice, in the future, to do this step programatically but for now…

Let me know if you’ve got any suggestions or comments. The technique for all of these utilities is a simple 3×3 moving window so this code might serve as a good template to develop other raster processing utilities… let me know what you come up with!