PerryGeo

FDO, GDAL and FME all seem to operate in roughly the same domain - Providing a data model, API and tools to translate between spatial data formats. Does anyone know of any good write-ups comparing/contrasting the features of these three libraries?

A few weeks back, I decided to take the plunge and learn the python bindings for QGIS 0.9. My first experiment was to implement a geocoder plugin. What started mostly as a learning experiment turned into something that might actually be useful!

The idea was to use web services to do all the actual geocoding work (the hard part!) and the delimited text provider to load the results into qgis. Right now it’s built on top of the Yahoo geocoder which is, IMO, the best out there.. very flexible about the input format. The geopy module is used to interact with the geocoding services so it could potentially support other engines such as geocoder.us, virtual earth, google, etc.

The user interface is very straightforward; enter list of addresses/placenames seperated by a line break, pick an output file and go. To be legitimate, you should also sign up for a yahoo api key, though the ‘YahooDemo’ key will work ok for testing purposes.

Here’s the install process (assuming you already have python, pyqt4, qgis 0.9, qgis bindings, etc. set up):

 svn checkout http://perrygeo.googlecode.com/svn/trunk/qgis/geocode
 cd geocode
 emacs Makefile # change install directory if needed
 sudo make install

This is just a rough cut and it’s my first attempt at using the qgis and qt apis so there are probably many things that could be improved upon. Ideally this plugin could:

• Parse text files as input
• Allow for a choice of geocoding engine
• ???

Feedback (and patches) welcome

If you want to get a linux/unix machine talking to your GPS unit, most likely you’ll be using gpsd. There are many great apps that build off of gpsd such as kismet and gpsdrive.

Installing gpsd on debian/ubuntu systems is as simple as

sudo apt-get install gpsd gpsd-clients

You should be able to connect your gps via serial port and start a gpsd server

sudo gpsd /dev/ttyS0

The gpsd server reads NMEA sentences from the gps unit and is accessed on port 2947. You can test if everything is working by running a pre-built gpsd client such as xgps.

This is very useful for situations where you need lower-level access to the gps data; for logging your position to a postgres database for example. The debian packages (and most others I’m assuming) come with gps.py, a python interface to gpsd allowing you to pull your lat/long from the gps in real time. This opens the door for all sorts of neat real-time gps apps.

import gps, os, time

session = gps.gps()

while 1:
    os.system('clear')
    session.query('admosy')
    # a = altitude, d = date/time, m=mode,
    # o=postion/fix, s=status, y=satellites

    print
    print ' GPS reading'
    print '----------------------------------------'
    print 'latitude    ' , session.fix.latitude
    print 'longitude   ' , session.fix.longitude
    print 'time utc    ' , session.utc, session.fix.time
    print 'altitude    ' , session.fix.altitude
    print 'eph         ' , session.fix.eph
    print 'epv         ' , session.fix.epv
    print 'ept         ' , session.fix.ept
    print 'speed       ' , session.fix.speed
    print 'climb       ' , session.fix.climb

    print
    print ' Satellites (total of', len(session.satellites) , ' in view)'
    for i in session.satellites:
        print '\t', i

    time.sleep(3)

… which gives you a simple readout to the terminal every 3 seconds.

Obviously there are much more interesting applications for this ( logging data to postgis, displaying real-time tracking data in QGIS via a python plugin, etc). But this is a good start for any python based app.

Edward Tufte, the outspoken guru of data visualization, has long been an advocate of clear and concise (almost minimalist) graphical representations of data. He’s got a lot of great ideas relevant to cartography (my cartography course at Humboldt State used his book “The Visual Display of Quantitative Information” as our text).

One of the coolest ideas are “sparklines” which he describes as “data-intense, design-simple, word-sized graphics”. Instead of standalone charts that are often placed on their own and separate from the text that discusses them, sparklines are meant to be placed in-line with the text and provide memorable, simple and contextually-relevant data to support the surrounding text. For example:

The US National Debt as a percentage of GDP increased during the Reagan and Bush presidencies but dropped off slightly during the Clinton administration .

Now of course I had to figure out how to produce these in python. Theres a great cgi application, written in python by Joe Gregorio, that does sparklines. I needed something that was abstracted away from the CGI framework, more of a proper python module. Replacing all the CGI-specific code was straightforward and I came up with a standalone sparkline python module ( View / Download the Source Code. ) The only dependencies are python and the python imaging library.

In the minimalist spirt of sparklines, the interface was kept simple. First you create a list of data values then simply pass the list to one of the sparkline generators:

import spark
a = [32.5,35.2,39.9,40.8,43.9,48.2,50.5,51.9,53.1,55.9,60.7,64.4]
spark.sparkline_smooth(a).show()

Or if you prefer a more discrete, bar-graph-style instead of a smooth line:

spark.sparkline_discrete(a).show()

There’s plenty of room for configuration. For example, in the national debt example above I wanted to keep the y axis at the same scale (instead of the default min-max scaling) and make each step 6 pixels wide:

spark.sparkline_smooth(a, dmin=30,dmax=70, step=6).show()

How does this relate to cartography? GIS typically takes a snapshot representation of earth, frozen in time. Since sparklines seem particularly good at representing change-over-time, it could be an interesting way to add a time dimension to a 2-D map. For example, instead of just displaying country polygons with labels, you could place a sparkline right under the label showing the population changes over the last century. It seems like it would be an ideal way to embed alot of useful information into a small map.

Anyone know of any good examples?

I got the chance to see Paul Hawken speak tonight in Santa Barbara. I knew him best as the author of Natural Capitalism which provided a great roadmap for integrating ecologically sustainable practices with the business world. This talk was based on his recent book - Blessed Unrest - How the Largest Movement in the World Came into Being and Why No One Saw It Coming.

The basis of this book is simple: that organically-developed, bottom-up, non-hierarchical organizations (which number in the millions according to his research) are now leading the world in many diverse areas of service. He describes these environmental and social justice organizations as the “immune system” of our societies; our response to destructive and corrupt habits perpetrated by those in power who are willing to compromise our future for short-term gain.

One thing that struck me about the subject was the importance of sharing information and ideas (as opposed to spreading an ideology). I thought one of the most interesting stories of the night was his description of how the meme of non-violent civil disobedience evolved… from Emerson, to Thoreau, to Ghandi, to Rosa Parks to Martin Luther King, Jr. At each turn of the story, there was someone (often unnamed but vitally important) who turned on each of these people to the ideas of those who came before.

Paul was eager to point out the role of technology in this inter-connected mesh of grassroots community organizations. He mentioned open-source software a few times and even gave a shout out to Ruby on Rails (which I gather was the backbone for his WiserEarth.org site focussed on connecting these diverse organizations).

It was a careful mix of optimism and pessimism; Paul was careful in noting the many severe challenges we’ve been handed but was confident that this bottom-up mesh of interconnected citizens can form a community strong enough to withstand anything that comes it’s way. In the end, his message was about doing what you love, connecting with others and standing up for your values. Sounds like good advice to me.

Don’t you just love getting CAD data into GIS! I received a .dwg file with study areas delineated as polylines which we needed as polygons for analysis purposes. And it wasn’t just one polyline surrounding each study area … there were hundreds of little line segments which outlined a couple dozen areas (what was this CAD tech thinking?) . Luckily each segment had a name to associate it with the proper area.

I found that ArcMap’s tools for doing this are painfully inadequate so I turned to postgis. After converting the dataset to a shapefile, the solution was simple:

shp2pgsql “study_areas.shp” areas | psql -d gisdata
pgsql2shp -f “study_areas_poly.shp” gisdata \
“SELECT BuildArea(collect(the_geom)) AS the_geom, name
FROM areas
GROUP by name”

Viola… a new shapefile with my proper polygons instead of CAD chicken scratch.

The worldwind java sdk has finally been released. It’s a neat SDK, well organized, easy to bring into Eclipse with some good examples to start hacking away.

The only problem is the examples are written in Java . If braces make you cringe but you still want to work with all the excellent Java libraries out there, you’ll want to take a look at Jython. Taking the AWT1Up.java code and porting a subset of the functionality to Jython was surprisingly easy and yielded much more readable code in my opinion. And the ability to manipulate objects at the interactive prompt is just so sweet.

View the Source Code

Setup is not too terrible:

1. Get a Java JDK (I’m using sun java 6)
2. Download and install Jython 2.2b2
3. Download and unzip the worldwind java sdk (ex: /opt/wwj )
4. Set your LD_LIBRARY_PATH variable to /opt/wwj
5. Set your CLASSPATH variable to /opt/wwj/worldwind.jar
6. Run jython wwj_demo.py

One thing that is a bit disappointing with the WorldWind SDK in general is the lack of support for rendering common formats. Maybe I missed something but I couldn’t get gpx or georss feeds working properly. It is version 0.2 so I expect support for GeoRSS and GPX to improve and for GML, KML, GeoJSON, Shapefiles, Rasters, WMS, etc to be included eventually.

Anyone else out there started playing with Jython / Worldwind yet?

I’d like to have some interesting excuse as to why I haven’t posted since last July. But I don’t.

I’ve since left my postion at NCEAS, started a new job at Geosyntec and have been keeping busy with life, love and the pursuit of happiness. Oh and GIS of course.

Anyway, I expect to be posting on a much more regular basis from here on (unless I get distracted again ).