Workshop on Using OpenStreetMap Data

 

I presented a workshop (or at least a live demo session) at the Society Of Cartographers conference with the rather vague open ended title of “Using OpenStreetMap Data”   –  “A tour of the various options for downloading and otherwise accessing OpenStreetMap data from a geo-data user’s perspective. Harry Wood will explain how to delve into the raw data structures using tools on the website and elsewhere, how to explore the wiki-style editing history, how OpenStreetMap’s unique ‘tags’ approach works, and some ways of manipulating the map data.”   At least that’s what I wanted it to be. It didn’t go entirely to plan (see apologies below)

I started by presenting some slides from my OpenTech OpenStreetMap developer ecosystem presentation which highlights the central role of raw geodata, and gradually builds up a picture culminating in this diagram (see above link for the full build-up and explanation)

Also a re-use of the slide explaining different levels of OpenStreetMap use which developers and data user organisations might consider.

Then it was on to the live demos touring around various different topics and tools. I don’t think I actually timed it well enough to get through all these things in either of the two hour-long sessions, but the following were things I intended to at least note briefly, if not doing a full-on demo.  I’ll resist the temptation to flesh this out with more text. Brief notes and lots of links is the best way.


Workshop demo notes:

 

Wiki and other documentation
Go to OpenStreetMap.org and click the ‘Documentation’ link.  http://wiki.openstreetmap.org  Search box usually works
http://help.openstreetmap.org has lots of info in Q&A format.
Map tiles
On OpenStreetMap.org right click, show image, to reveal a tile URL e.g.
Format is {base tile server URL}/zoom/x/y.png
Using tiles as a slippy map. OpenLayers example:
Download options
Planet download  http://planet.openstreetmap.org 22GB of compressed XML (300Gb uncompressed)
“Extracts” at the country level e.g. http://download.geofabrik.de/osm/
and “Metro extracts” http://metro.teczno.com
Shapefiles from geofabrik and cloudmade.
Garmin img files.
API map calls.
Nodes, Ways, Relations, Tags  – Seen within the following, and within the raw XML
Simple views of the data
‘Data’ viewOpenStreetMap.org Edit tab drop-down. “Browse Map Data”
Potlatch 2 – The flash editor
JOSM
Download a jar file and double-click
Tagging
Folksonomy approach.
Osmosis
http://wiki.openstreetmap.org/wiki/Osmosis  Small tool. Extract to install. Requires java again.
To extract central London from the greater London area metro file:
./osmosis-0.41/bin/osmosis \
   --read-pbf london.osm.pbf \
   --bounding-box top=51.5398 left=-0.2018 bottom=51.489 right=-0.0534 \
      completeWays=true \
   --write-pbf central-london.osm.pbf
Takes several minutes, but here’s a quicker bounding box extract of just UCL area
./osmosis-0.41/bin/osmosis \
   --read-pbf central-london.osm.pbf \
   --bounding-box top=51.523351 left=-0.133344 bottom=51.522196 right=-0.131353 \
           completeWays=true \
   --write-xml small-UCL-extract.osm
Wont run this one, but here’s how you would use unix piping to unzip and rezip input and output bz2 files
bzcat london.osm.bz2 | ./osmosis-0.41/bin/osmosis \
   --read-xml file=-\
   --bounding-box top=51.523351 left=-0.133344 bottom=51.522196 right=-0.131353 \
           completeWays=true \
   --write-xml --write-xml file=-\
     | bzip2 > small-UCL-extract.osm.bz2
Get just the ways representing large buildings tagged shop=supermarket:
./osmosis-0.41/bin/osmosis \
   --read-pbf central-london.osm.pbf \
   --tf accept-ways shop=supermarket \
   --tf reject-relations \
   --used-node \
   --write-xml supermarkets-ways.osm
Get just the nodes where a shop=supermarket was mapped just as a single point
./osmosis-0.41/bin/osmosis \
   --read-pbf central-london.osm.pbf \
   --tf accept-nodes shop=supermarket \
   --tf reject-ways \
   --tf reject-relations \
   --write-xml supermarkets-nodes.osm
Merge two .osm files together
./osmosis-0.41/bin/osmosis \
   --read-xml supermarkets-ways.osm \
   --read-xml supermarkets-nodes.osm \
   --merge \
   --write-xml supermarkets.osm
“all-to-nodes” feature of osmconvert is handy simplification
./osmconvert supermarkets.osm --all-to-nodes >supermarkets-all-nodes.osm
(output shown in JOSM at each of the above steps)
osmconvert can convert this file to CSV
./osmconvert supermarkets-all-nodes.osm --csv="@lon @lat name"
Can imagine further uses of this simplified data e.g. “find my nearest supermarket” phone app
Other tools: osm2csv.rb, osmium,   osm2pgsql  (import to PostGIS)
XAPI
example URL (supermarkets in central london) :
Diffs
Consume updates from the community.  Osmosis replication.
Meta-data
browsing editing history.
– via data layer on the web
‘recent edit’ displays (itoworld) http://www.itoworld.com/map/group/20
full history dump – even bigger bulk download!
Phew! That’s more than enough for an hour.

Apologies

I was a little wrong-footed from the start, and I need to apologise for this. I couldn’t get my laptop onto the internet. I could blame the over-complicated university wifi, but it was my own fault for turning up 2 minutes before running the workshop… which was my boss’ fault for organising a lunchtime client meeting immediately beforehand, then leaving at the time when I needed to be leaving. Anyway, I was able to fuddle along using two different computers to demonstrate most of the intended topics. …Oh and by the way, in all the confusion I also managed lose my apple power charger (white square thing). Anyone got it?

I also realised from the start that my planned demos involving command-line XML manipulations, were going to be pitched poorly for a least some of the less techy audience, so I dwelled longer on more basic topics for their benefit… which may have bored the more techy folks.  It was a bit of a mixed crowd, but that’s all part of the fun of the Society of Cartographers, and hopefully everyone learned something!

One thought on “Workshop on Using OpenStreetMap Data

  1. Hey Harry,
    No apologies needed. I really enjoyed the workshop. It is great to meet the guy that has helped me via forums with many osm2pgsql and OSM data troubleshooting problems.

Leave a Reply to Nat Evatt Cancel reply

Your email address will not be published.