My SOTM 2017 talk: Diagrams of OpenStreetMap


I gave a talk at State Of the Map 2017, the official OpenStreetMap annual conference, which took place in Japan last month.

“The Diagrams of OpenStreetMap” is a tour of a various diagrams illustrating the project, but focussing more on my own diagramming efforts, including quite a few new diagrams I produced especially for this talk.

I would have mentioned in my talk, but I wasn’t going public with it at the time: I’m going to be available for new work opportunities soon. Maybe I can find something map related? We shall see (suggestions welcome). In any case I’ve been involved in OpenStreetMap, mostly as a volunteer, for over a decade now, and so I’m well placed to try to capture some “big picture” overviews of the project… in diagram form! That’s what I’ve tried to do here.

Watch the video on the conference site

Slides as a LibreOffice .odp file (7.3Mb)

But the following are slides and transcript as one big long blog post. In total there’s a whopping 61 slides (I was talking through them quite quickly). I’ve skipped over some animation step slides to reduce this down a bit, but still a lots of slides:

 

Jump to slide:
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61


Slide 1

 

 

This talk is called “Diagrams of OpenStreetMap”. I started thinking about this after my previous 2014 State Of The Map talk.

 
See the rest of the slides

OpenStreetMap Notes in MAPS.ME

I made a tool to scratch an itch I’ve been having recently. I wanted to see OpenStreetMap notes within the MAPS.ME app, so I made a thing to fetch and generate…

OpenStreetMap notes as KML

This let’s you grab all the notes in an area and download them as KML. Such a download might be useful for various things, but mostly the idea is to easily load notes into MAPS.ME

It works nicely if…
(A) you have the MAPS.ME app on your phone, and…
(B) you open the above link on your phone  (Or this QRcode)

The KML download will (after some confirmation prompts) be handled by the MAPS.ME app, and the result is…   a shed-load of red bookmark icons:

MAPS.ME notes map

It will fetch up to a limit of 1000 notes for an area of your choosing, appearing as a layer of “bookmarks” which you can show and hide (third icon on the bottom there).

On a bookmark, swipe up to see the detailed description and you get the full note description plus any other comments people have added:

MAPS.ME Note comments

I have control over how that gets formatted, so it’s something I could tweak/improve

What is MAPS.ME?

You don’t know? It’s an app which you can install for iPhone or for Android for viewing maps …specifically OpenStreetMap, but it’s worth having even if you’re not a big OpenStreetMap enthusiast, as an alternative to the “maps” app on your phone. You might use it occasionally e.g. when the maps app doesn’t have the detail you want. MAPS.ME requires an initial (quite big) download of an area, and then it all works offline. Great for going on holiday somewhere.

What are “notes”?

OpenStreetMap Notes allow people (anyone. without even logging in) to report a map problem or an omission on OpenStreetMap. So if you don’t fancy learning how to edit OpenStreetMap (we try to make it very easy, but …takes a bit of learning) then instead you can effectively ask somebody else to make the edit, by simply leaving a note. However it’s important to position the note precisely add as much description as you can. Preferably even link a photo of the thing which is missing/wrong.

As a mapper you will understand the importance of this when you come to try to clear out the open notes in an area. In London it feels like most notes are pretty unclear and poorly described, but of course we’re generally only looking at the notes which are left open. It’s quite common for somebody to write just a few words, seemingly reporting the name of a missing thing, but if that’s the level of effort somebody is willing to go to (not even constructing a sentence) then the data doesn’t feel reliable enough to do anything with, other than to leave open and go survey. And there’s various other types of notes which lead to this same conclusion. Going to take a look on-the-ground is going to be the way to solve the note.

And going to look on-the-ground is really the idea behind this. Because I use MAPS.ME a lot while out and about, for everyday navigation, I’m hoping having the notes in there will help remind me to go take a look

Open House rails developer

[Update: They’ve now hired a rails developer. Thanks for the responses! Still opportunities to help I’m sure. I will to try post an update on how to get involved]

I wanted to help promote this Ruby on Rails developer job at OpenCity which I think will be a really interesting thing for somebody, maybe a junior rails developer. Know anyone?

If you’re a londoner you’ve probably heard of “Open House” an event taking place each year where, for one day, you can take a look around lots of interesting buildings, for free, which are often closed to the public the rest of the time.

The Open House event is put together by a not-for-profit organisation called OpenCity, in their office near Aldgate. From there they organise this annual architectural bonanaza, coordinating hundreds of people (volunteers and building owners) with lots of careful planning …and some IT challenges.

opencity-diagram-currentopencity-diagram-1st-stage-rails-app opencity-diagram-big-rails-app

Being an interesting and fun not-for-profit organisation, I have ended up volunteering a bit of my time with them. Me and another volunteer have been planning a rebuild of a database system they’re using internally, which will later ripple up to some cool improvements on their public facing website.

To me this looks like a juicy challenge and a fun organisation to be helping out, so I’m hoping we can find a suitable available developer who feels the same! I’m not available myself, except in my spare time. I plan to be dipping in on this project from time-to-time, so I would be working with this person a little bit.

The project planning is at a very early stage, but I’ve been drawing the above diagrams which show… (left-to-right)

  • Their current set-up
  • An initial milestone introducing a rails app for their buildings database
  • And a final situation with websites and database consolidated

Hopefully in the end we will have managed to consolidate things and four different websites (or four different user journeys/permission levels within the same website) will be served by a lovely new rails app!

OpenCity are very open to our ideas, and one thing we’ve suggested, is to do this whole thing open source on github, as a way of being open to contribution from any other developers who fancy helping this organisation, but this is a pretty big job which will need somebody on it full time. Hopefully quite an interesting challenge for somebody! Please pass this on to anyone who might be interested.

Thanks Chain Reaction Cycles!

Look how happy our baby is with his new little bike:

baby-bike-crc

It arrived in a box yesterday along with a letter from Chain Reaction Cycles:

chain-reaction-baby-letter

“Dear Baby Wood,

We noticed that your daddy got a nice new bike delivered from our website and you were excited by this, so we wanted to send you a bike of your own. Hopefully you will be big enough to start riding it soon.

Chris R – Tech Team Leader”

 

How lovely!

They did all of this in response to a tweet I had sent a few days earlier, which was proving rather popular:

chain-reaction-box-baby-tweet

So that was some good tweeting!

31 “likes” on twitter (so far). That’s after Chain Reaction retweeted it. It seems like the likes came from random customers and cycle fans up and down the country. 31 likes seems to be well above average for @Chain__Reaction‘s twitter stream, and 31 likes is very good going by my standards. My best performing tweet ever in fact! So much for my years of tweeting witty and insightful commentary on tech and maps and things. They all just got beaten by a picture of my baby!

Anyway, what a great surprise to receive! It was addressed to “Baby Wood” which caused some amusement when the courier showed up. I didn’t open it at first in case my wife was planning some sort of surprise I wasn’t supposed to know about. But no! It was surprise from Chain Reaction Cycles.

As you can see, the baby can be balanced upon the saddle just about, but he’ll need to grow a bit before riding it properly. …Or maybe I should get him training early. See if he can learn to cycle before he learns to walk!

Keyboard Letters Game

I made an online game for my 8 month old baby to play. If you are a baby you might like it too! :

>>> Keyboard Letters Game <<<

Maybe I should come up with a more imaginative name for it, but the idea is, he can bash the keyboard and big colourful letters pop up on screen. It also reads out the letters (put your sound on). Here is a video of him in action:

keyboard letters video

[Update: source code is on github]

Our little boy is 8 months old now. For quite a while he’s been interested in my laptop. He’s interested in any new objects, but I think he’s also seen me typing away on it, and wants a bit of that action. He impatiently tries to reach for my laptop whenever he’s nearby.

I realised he’s a bit young for this still. As you can see in the video, he’s whacking the keyboard quite violently with his whole hand. In fact we have a toy with a just one big button, and he’s still developing the coordination needed to press that, so he definitely doesn’t get the idea of pressing individual keys on the keyboard, but he does seem to be making the link that keyboard whacking makes sound and colourful things happen. Unfortunately he’s also very good at somehow finding weird and wonderful keyboard shortcuts I never knew existed, for dropping out of fullscreen mode, and opening settings etc.

For his age, I’m getting some ideas for simpler games I could develop which just bring light colour and sound based on approximate location on the keyboard he’s whacked.

When he’s a bit older I imagine this might help him understand typing and also letters of the alphabet, particularly as it reads out the letters. Mind you, I’m told they don’t teach the reading of the alphabet in the same way any more in UK schools :-O   Do I need to make a “synthetic phonics” version of this?

I actually made this game for the first time when I was helping out with IT in a junior school. I must have been about 15/16 years old at the time, and I made all kinds of experimental things in BASIC running on Acorn computers (odd things we had in schools at the time). So this is a reimplementation of 20 year old idea!

It’s nice that we can do this kind of thing on the web these days, although doing this with javascript is probably quite a messy bodge-job with nasty browser compatibility considerations compared to doing it in Acorn BASIC. It makes me wonder how I would get on as a 15 year old learning to program in modern times. I don’t remember exactly how I did speech synthesis 20 years ago (system call-out to do a *SAY "HELLO" command perhaps). On the web I didn’t actually know this was possible. On a whim I googled “speech synthesis HTML5” thinking “Nah surely it’s not…” but it is! Speech synthesis is available for calling in javascript on modern browsers! Might have to have some more fun with this!

…if my baby doesn’t destroy my laptop first

Leaflet Geolocation error: Only secure origins are allowed

I described some reasons to switch to HTTPS on my website. To be completely honest though, I didn’t finally get off my ass do that for any of those good reasons. I did it because I was building a map thing which requested browser geolocation and I noticed geolocation stopped working in chrome.

I’ve seen this deprecation warning a few times:

“getCurrentPosition() and watchPosition() are deprecated on insecure origins. To use this feature, you should consider switching your application to a secure origin, such as HTTPS. See https://goo.gl/rStTGz for more details.”

But somehow didn’t take it seriously. But yes. New versions of chrome won’t do geolocation unless it’s a HTTPS site. See this for yourself with this very basic geolocation test page on w3schools (which is http). Doesn’t work in chrome.

The javascript console still only shows it as a deprecation warning not an error, but if your web application was relying on this…  it broke.

(Update for Aug 2017) Firefox v55 is going with this lock-down too. It says “Geolocation error: User denied geolocation prompt” as a popup, and in the console “A Geolocation request can only be fulfilled in a secure context.”

Any sensible application should probably be watching out for failure cases with geolocation anyway (see later examples for handling errors), but even so I find it a bit surprising that any old websites using geolocation across the web will be broken. There’s a bit more info on this google developers page

If you use LeafletJS, there’s a map.locate method which presumably uses the same method internally (navigator.geolocation.getCurrentPosition), but leaflet also detects the Chrome failure and pops up a different error message…

“Geolocation error: Only secure origins are allowed (see: https://goo.gl/Y0ZkNV)..”

If you use chrome you can see this on my geolocate example (http) here:

http://harrywood.co.uk/maps/examples/leaflet/geolocate.view.html

…and    *Trumpet noise*   see it fixed with the newly available https URL:

https://harrywood.co.uk/maps/examples/leaflet/geolocate.view.html

HTTPs on this site

I made harrywood.co.uk run on HTTPS recently (optionally. It works on both http://harrywood.co.uk and https://harrywood.co.uk). Quite easy to do, and free using letsencrypt

https

Why encrypt harrywood.co.uk?

On the face of it there’s not much point. This is mostly just a straightforward read-only website. Not much scope for bad people to be snooping anything interesting. No passwords or credit cards or anything. General “tracking” doesn’t seem particularly problematic either. Who really cares if somebody can track the fact that you’ve been visiting these sweet innocent innocuous blog posts? Well…

Some types of commercial web tracking only tend to get creepy when they happen in bulk. The evil corporate advertising machine won’t learn much about you from knowing you read a blog post on harrywood.co.uk, but it might start to know you pretty well if it knows this and the previous thousand websites you visited. Encryption throws a spanner in the works for some types of tracking.

Government tracking by intelligence agencies, is also thwarted by encryption (more so probably). They would also like to intercept your browsing traffic to get to know you with their big evil AI. Now sometimes I think it’s fair enough for governments to do a bit of anti-terrorism targeted snooping, but the trouble is it’s too easy for politicians to make that simple-minded argument. The flipside is a subtle future threat of eroded freedoms. That’s tricky, and in general I don’t trust politicians to weigh it up properly. We can use technical measures (encryption!) to help things move in a more freedom preserving direction.

Tracking is a numbers game, done across many websites, and equally encryption as a counter-measure is more effective if we encrypt many websites. If we start to be able to browse a significant proportion of the web in HTTPS, even right down to piddly little websites like this one, then we’ll be getting somewhere. As a result it’s becoming recommendation and slowly a sort of groundswell of expectation on webmasters to do this. It’s slow to get lazy webmasters like me to do something like this, but …well now’s the time for harrywood.co.uk (Who knows? One day I may actually work on updating the content!)

Encryption helps protect against password snooping security issues. harrywood.co.uk has no user passwords, except…  my own password for logging in to write blog posts. I’ve probably used this from public wifi access points in the past. Slapped wrists for me. But now I guess I can be a little more relaxed about that. Speaking of wifi, wifi javascript injection (attacks or just crappy advertising) seems like a nasty problem. Are we safe using any wifi these days? Well we’re a lot safer from this when browsing HTTPS sites.

Free iOS games

Here’s some iOS games I’ve found enjoyable. These are all FREE.

angry-birdsObviously Angry Birds is a good place to start if you’ve not played any games on your phone. A blockbuster success for a reason. Quite addictive, and broad appeal (meaning they’ll draw you in, even if you’re not the kind of person that enjoys games). There’s also more of the same with several variations on the theme: Angry birds seasons. Angry birds star wars. Angry birds rio. etc etc.

I guess Candy Crush is similar. That one’s so mainstream my wife plays it. Personally I’ve deliberately denied myself that time-waster. In general I’m trying to avoid the super-mainstream blockbuster apps, but…

dotsDots is the stylish minimalist version of candy crush. Same idea. Swiping to match up lines and clear things tumbling down from above. But yes, less cutesy more minimalist graphics.

line-upLineup Puzzle. Make lines from different shapes, very similar to tetris, but without the blocks tumbling down from above (is this all sounding abstract enough for you)

amazing-thiefAmazing Thief. Another super-dinky little game. In fact this “Ketchapp” developer seems to specialise in insanely simple little free games. I’ve tried a lot of them, but this one was my favourite. I don’t quite know why I played this one until I managed I score of 22. Maybe because the single tap control is good for playing while crammed on a crowded tube train.

little-craneThe Little Crane That Could. Ever wanted to pull those levers to control an articulated crane? This 3D crane simulation is remarkably entertaining. I think it’s the realistic physics simulation that makes it. The free version only gives 5 missions. Even I have been tempted to pay for more.

airplaneAirplane! So speaking of physics I tried a few flight sim free apps. This one didn’t have particularly good physics, but it kept me entertained for quite a while for some reason. Maybe the choice of planes and the challenge of enabling new ones.

f18-carrier-landingF18 Carrier Landing. This one has better physics and more interesting controls. Weirdly they went to the trouble of developing this nice flight-sim engine and then made the game mission extremely limited. You just land on an aircraft carrier. That’s it. It’s like they forgot to make the rest of the game. But randomised weather conditions and start locations kept me entertained.

I can’t say I’ve done a systematic review of lots of free apps by genre. This is just some apps I’ve stumbled across and which turned out to be pretty entertaining (sometimes against my first impressions).

Playing silly little games on my phone is a guilty pleasure of mine. I’ve generally only ever installed free games, because… well, long story short, I’m a cheapskate. But they are very hit & miss. They can be a bit rubbish or not working at all for whatever reason. Increasingly they can have such unbearable advertising bombardments, in-app purchase nags, or sign-up demands, that I don’t really get as far as figuring out if the game is any good. Sorting the wheat from the chaff is a problem. Hence I thought this list might be useful for others.

I’d like to list some free and open source examples, but it seems open source isn’t a big thing for iOS developers. There’s very few working games on the appstore (as opposed to game engines / ideas) where you can also find the source code. I thought I’d installed Lumio off the appstore (quite a nice little game) but can’t find it on there any more.

I’m actually getting around to writing this finally now that I’m ditching my iPhone (and all these games) and swapping to a new android phone! I imagine I may be able to write a similar list but just for free open source games on android.

Vote remain

This sums up the EU referendum pretty well for me:

“If you vote leave it doesn’t necessarily mean you’re xenophobic,
but if you’re xenophobic you’ll vote leave”

I read this sentence somewhere else, but with the word “racist” instead of “xenophobic”. I think “racist” feels too strong, but I do think there’s unfortunately quite a high proportion of the UK population who are mildly xenophobic. They don’t like foreigners that much. And that is pretty much all this EU referendum vote will come down to.

There are of course lots of other important issues being debated.

Take trade for example. I see the remain camp saying that a brexit would be totally disastrous for trade, and then the leave camp saying that it’d be fine, we’ll easily strike up trade agreements. I find myself thinking the truth is probably somewhere in between. There’ll be a lot of admin and hassle (better have a good reason for it) but in the end we’d get trade agreements in place and we’d be trading with Europe as we always have.

And there’s lots of similar tit-for-tat debates where the campaigns have been quite polarised, with scare-mongering and lies from both sides.

So I started out thinking, let’s leave aside these smaller short-term issues and think about the big picture: The EU represents unification. Working together closely with our european neighbours. To me that’s very compelling, especially in a world where increasingly Russia and China are calling the shots. Let’s be unified. Let’s be a European super-power. But the leave campaign says, in an increasingly turbulent world we need more control over our own affairs to be able to better cope with whatever the world throws at us, acting in our own interests. I can see that point. I can see why others might be persuaded by that. Personally on these broad big picture viewpoints, I’m more for unification.

So I was always taking a remain view, but I recently realised I was watching/reading too many debates and losing sight of the real driving force of this referendum. This makes me feel very strongly in the remain camp with the realisation that, if the country does vote to leave, it’ll be for a terrible reason. I’m convinced the majority of people voting to leave are not even thinking about the issues. The only reason the EU referendum has ever even been a thing, is because a lot of people in this country are mildly xenophobic.

For some people this is because they take in too much tabloid nonsense (particularly on the massively over-inflated non-issue of immigration), some people have fooled themselves that it’s the right way to be “patriotic”, for others it’s just a failure to reign in their innate distrust of foreign people. Fear/distrust of people who are not like us, is something which exists in all of us. A perfectly natural feature of our monkey brains. We’re more likely to learn to take our xenophobia and set it aside if we work with foreign people, or we do a lot of foreign travel, or we live in a part of the country which is more cosmopolitan. But I worry that many people when they head to the ballot box will be listening to this nasty little voice, this mild xenophobia, in the back of their mind, and they’ll vote leave for no other reason.

I don’t know if this is a good way to persuade people. We saw as UKIP were taking votes in the general election, that nobody likes being told that they’re racist. Maybe telling someone they’re mildly xenophobic is not much better. But what this does mean, is that a general background level of xenophobia is swaying the vote towards brexit. What a terrible reason to take a decision like this. It means all right-thinking people, even if they find themselves on the fence on the EU issues, must make themselves count as an antidote to this.

vote remain

BRemain

UK passport photos for 39p

photo_boothI recently had to get some UK passport photos printed for my baby. This has extra fun issues, like getting him to look at the camera, and not having a hand visibly holding him. But even for adult passport photos…  I have a nice camera. It feels like paying a £5 for a photo booth is wrong. So I have used the following approach …because I’m cheap like that.

At boots you can print a single photo for 39 pence, and a single photo is big enough to fit eight little UK passport photos within it. Bargain! We just need to prepare a picture like this to print:

passport-photos-baby-small
(There’s our little lad!)

The trick is to get the pixel ratios right. The other trick was to realise that boots photo machines crop photos a little bit at the edges, and they actually automatically crop more if you give them an image with a lot of blankness, hence the decorative leafy border on this image to prevent this. This skinflint Yorkshireman has done the trial and error so you don’t have to!

Passport photos template image <<< (right click ‘save as’)

So you might make use of this file as a template, and edit it to put your own passport photo image, but you’ll have to do some clever resizing and cropping and positioning using image editing software. The overall photo file here is 3264 x 2448 pixels. Your mugshot photo image will need to be sized 738 x 949 pixels, but don’t forget there’s rules for how big the head should be within a passport photo. So this means within that image you need the head to be ~660 pixels tall. In fact it’s easiest to take that as the starting point.

So the steps are:

  • Take your photo
  • Make a copy of the file
  • Resize it so that the head is 660px tall
  • Crop it to frame the picture nicely and to exactly 738 x 949 pixels
  • Save that
  • Open the template image and copy your image onto it
  • repeat for all 8 positions

Finally put that image on a USB key/SD card and take it to boots. When you come to print, select “6×4 From: £0.39”

And there you have it… 8 passport photos for 39 pence!