I’m no longer twitter domesticated

It’s a weird time in technology and the web. It feels like a couple of big shifts are happening all at once. One of them is a big rise in “enshittification”, most noticeably for me with the death of twitter.

It’s pretty surprising watching twitter die and, I’ll be honest, it’s upsetting. I allowed twitter to become a bigger part of my online life than I should have done, and now it’s going down the toilet in a shift that feels quite sudden, given that I’ve been on twitter for about 16 years. Throughout 2023 I found myself gradually swapping from twitter to mastodon. Admittedly it wasn’t a very deliberate choice. I still go on both, but these days I think about posting to mastodon first and I’ll copy the post into twitter if I can be bothered.

Why is twitter going down the toilet? (I refuse to call it “X” by the way) It took me a while to notice the problems, and maybe it was only the contrast with mastodon that helped me see it more. People talk in dramatic terms about nazis posts, which I never really saw, but there’s a shift in “the algorithm” of the feed. Mastodon only shows stuff from people I follow (and their re-toots). When I first played with mastodon a couple of years ago, it felt similar to twitter, but now twitter has shifted. It’s not even a subtle shift any more. My twitter timeline has very few posts from people I follow. Instead it’s full of clickbait videos and other “engaging” posts from non-followees. A lot of ads too of course.

This algorithm shift is also noticeable with less engagement on my tweets. I see fewer likes and retweets these days. My tweets are hopefully not too dull, but they’re not exactly carefully engineered to “do big numbers”. They’re in a medium-interesting middle-ground which now seems to mean they go ignored more. And, before you roll your eyes at my dreadful narcissism, I know this is not just me. Others have said the same. If you’re guilty of paying attention to how many likes and retweets you get, then you’re probably noticing fewer of them these days. Meanwhile the tweets we do see, from strangers who we’re not following, do often appear to be engineered for engagement. All the click-bait tricks. This includes posing questions where we don’t get to see the answer until we click. Video tweets usually do the same. We see the intriguing freeze-frame. We don’t get to see what happens next until we click.

How twitter made me an angry cyclist

Then there’s the outrageous statements, or the statements which we know to be false and it’s hard to resist replying (or at least clicking to make sure that somebody has set them straight. Invariably hundreds of people already have). Just another kind of click-bait, but I think this one gives rise to the idea that the algorithm is actively evil. The outrageous statements can be on matters which are innocuous yet somehow engaging, but of course it’s also on serious matters of politics, culture wars, nazis versus black-lives-matter etc. I imagine this is where the “twitter is full of nazis now” idea comes from. I think it depends what you engage with. Also I think a lot of people look at what Elon Musk is interested in and maybe extrapolate that as a statement from twitter. But by even looking at that, the algorithm begins to steer you onto these topics.

From my own perspective I really noticed this, not on nazi topics, but on cycling related tweets. I follow quite a few keen cyclists. It’s not a big obsession for me although admittedly it might sound strange to some people if I say that I am sad to no longer be cycle commuting since I started working from home. The algorithm has really pounced on this. It bombards me with videos of cycling road accidents (because I have to click to see what happens!) and people airing rather extreme pro-cycling or anti-cycling opinions. I do click on this stuff. I can feel the result is that twitter is making me angry, to the point where I feel I’m being recruited as foot-soldier in the war between cyclists and motorists. The really bad thing is that I can feel it effecting my psychology and spilling out into the real world. On the rare occasions when I do get on my bike these days, I feel anxious and angry at all the motorists around me, ready to go out and get into a confrontation with somebody. …until I calm down and remember the truth …the reality …the big cycling secret. It feels unfashionable to say it these days because nobody finds it exciting or anger-inducing but the secret is: most other road users are considerate, and the vast majority of cycle journeys, even in central London, are enjoyable and pass without incident. How boring.

You would never get that from my twitter timeline, and this, more than anything, has made me aware of the nature of “the algorithm”. It cuts across all topics of course. In politics I used to think a big problem was the “echo-chamber” effect. But nowadays we’re presented less with our friends agreeing with us, and more with a weird mixture of strangers agreeing with us, plus strangers with unhinged extreme opposite opinions to us. It’s not mild-mannered strangers who vote the other way. It’s people posting short ridiculous caricatures of political opinions. Extra engaging and extra anger-inducing.

Profit-driven enshittification

I think people tend to attribute too much malice to the algorithm. Nobody at twitter sat down and said “how can we make all our users more angry about stuff?”. They just wrote an algorithm driven by clicks. If we engage with something, we get shown more of it. Maybe the anger that emerges from it, is in some sense an expression of human nature.

But of course this algorithm is controlled by twitter, and it’s a shift from what went before. Previously we saw more posts from friends, and fewer ads. But maybe we still attribute too much malice because after all, it’s just about $$$. Blame and outrage is directed at Elon Musk. I’m not here to defend him (the guy’s clearly a bit of a dick, and I wish he would stick to rockets and electric cars) but I can well imagine all that really happened is, he told the twitter staff to take urgent action on profitability. Sort of fair enough I suppose. They already knew about various levers they could pull to make more money. Musk told them to crank the profit levers up to max, and that’s the effect we see. No big nefarious purpose. Just more ads, and more shovelling of engagement fodder posts instead of the (apparently less engaging) posts from our followees.

We call this effect “enshittification” thanks to Cory Doctorow, who writes excellently on the topic. But this ramping up of ads and other money-making patterns has limits. I’m sure the staff at twitter have worried about whether they can get away with pushing the profit levers, the enshittification, up so high. Will users put up with it? It all depends on the degree of lock-in we feel. In this post, Doctorow explains

“An enshittification strategy only succeeds if it is pursued in measured amounts. Even the most locked-in user eventually reaches a breaking-point and walks away, or gets pushed”

“the strong incentives for product managers, executives, and shareholders to enshittify results to the precise threshold at which users are nearly pissed off enough to leave, but not quite”

From my perspective twitter has crossed that threshold. I know plenty of others who (wisely it turns out) had a lower tolerance and left twitter earlier. And in my free-software loving communities such as OpenStreetMap, I know folks who never even touched twitter in the first place (who no doubt read this kind of blog post with a smug sense of “told you so”). Of course quite a few people didn’t leave, but I guess the fact that it lost 72% of its value is strong evidence that Twitter has done something wrong, and that something is “enshittification”.

The word “enshittification” is taking off as a way to describe the place we’re at with a lot of tech at the moment. Actually it seems like a very clumsy word for it. I do prefer Rohan Kumar’s suggestion of “User Domestication”. Like animals which used to run wild and free, but which gradually become dependant on their masters, users are eventually herded into enclosed online experiences where the tech companies can tame us, pacify us and then exploit us.

Twitter user domestication (Sheep at South Molton Livestock Market)

It’s not a surprise to me that, as a profit-driven company, twitter would start to seek that profit in ways which make things worse for users, or that the lock-in network effect comes to work against us in that circumstance. I was a bit surprised by how dramatically they have yanked on the profit levers to enshittify twitter over a relatively short timespan.

It takes a lot to escape user domestication, but the cattle are bursting out of the twitter enclosure right now and running wild across the prairie. Or at least there’s suddenly a lot of people looking around for alternatives.

Seeking new pastures. The move to mastodon

One way the cattle analogy doesn’t follow, is that there’s no real harm in signing up for other services to try them out. You can join the herd in many new cattle enclosures at once. The only question is whether you allow yourself to become domesticated. For example I have signed up and created my profile on “threads”, the new Facebook/meta twitter alternative, even though this seems like an utterly asinine choice for a new twitter alternative if you think in terms of hoping to avoid future exploitation as a domesticated user. Why on earth would you put down roots on a social network closely tied to Instagram, the very flag-bearer of the engagement algorithms we were trying to get away from? There are alternatives, including some operated by quite small friendly looking tech start-ups. No harm in trying them out, but… there is a better way.

Mastodon is very twitter-like but free and open source software, and decentralised software. A network hosted across many “instances”. I won’t go into the full pitch here, except to say that it’s a smart alternative to the depressing inevitability of enshittification we see on any for-profit hosted service. I recommend it.

However… I think the mastodon community (who seem to do a lot of meta-posting about how wonderful mastodon is) is a bit too self-congratulatory sometimes.

On-boarding of new users is confusing just because you’re immediately faced with this paralysingly important choice: which instance to join? I don’t think it really matters that much, or at least I don’t think it should matter, and people should feel free to post on topics unrelated to the topic focus on their instance (the etiquette is a little unclear but hopefully most folks feel the same). But the choice of instance does matter when it comes to the usual service hosting type questions. It made me laugh when people first started crowing about mastodon saying things like “because we don’t trust twitter with all our micro-blogging content” …the solution is to move to some server our friend’s friend has spun up in their free time? This even pre-dated the ability to migrate accounts between instances, but even with that ability, the reliability of hosting (the idea that my instance might disappear suddenly) is still a worry. Relatedly there’s the question of funding. People donating, or talking about donating, to their mastodon hosts. Coming from the twitter-hosts-millions-of-users-for-free model, this feels a bit fragile, and perhaps unlikely to be able to receive those millions of users.

Many of us moved from twitter and followed eachother initially using tools such as “debirdify” or “fedifinder” to slurp our follow-lists from twitter (This is based on mentioning your mastodon account in your twitter profile, so be sure to do that) This was a good little cheat that meant mastodon didn’t really need to build the community from scratch. Even so, it’s obviously a sub-set of my twitter followees who have made this move. To me it feels like a more tech-savvy sub-set. Mastodon is like in the early days of twitter. That’s nice in a geeky clique sort of way. Mastodon enthusiasts will say we don’t want to attract all of twitter onto the platform. But I would like my less tech savvy friends to be on there, and to me the mastodon experiment hasn’t really succeeded unless/until it can go mainstream enough for that to happen.

Decentralisation offers a lot of solutions, but mastodon is not guaranteed to succeed. I think it’s too early to pat ourselves on the back and say we’ve successfully used mastodon to free everyone from twitter domestication.

But having said all of that, mastodon is better than the enshittified twitter experience of 2023 / 2024. I am feeling a lot less twitter domesticated these days. Time to follow me on mastodon instead!

Job at Vivup

vivup

I started working at Vivup (vivupbenefits.co.uk). Actually I started there approximately one year ago, so in order to keep this blog bang up to date as ever… time to tell you about this!

Vivup helps organisations provide benefits to their employees. We have many large government clients such as NHS trusts, local councils, police forces etc, and increasingly bringing in private companies too. The benefits on offer are many and varied. One part of the website is like a shopping site with all kinds of products available as a “salary sacrifice” purchase, paid for over many months. There’s lots of bicycles available via cycle-to-work schemes, and there’s store discount cards, and all sorts of health & wellbeing offerings. If these benefits sound good, you’ll have to arrange to sign your whole company up to Vivup!

As you can imagine, it’s quite a big complicated website, and it’s a rails app! We have a tech team working on various aspects, including people focussed on the mobile apps, and the separate .NET bits. We also have people who are a lot more front-end focussed. They’re migrating more and more of the site to use React on GraphQL. Our sub-team’s focus is on the rails coding. There’s back-end aspects, the database and API integrations, plus a big back-office admin interface built with ActiveAdmin, and lots of interesting workflows and dataflows. To deliver improvements to all of this, we have quite a rapid pace of projects running in parallel, which is taking some getting used to. We also have a bunch of QA people, which is something I’m not used to. Quite a big tech team in total.

But the wider organisation feels big compared to previous jobs. I think in my previous big organisation jobs (e.g. 3mobile, B.P. & Alstom) I was in tech project teams isolated from the rest of the organisation, whereas at Vivup I’ve been involved in occasional bits of collaboration and communication across the organisation. Also it’s small enough that the whole organisation gets together on a quick video call each Friday, and for face-to-face social meet-ups three times per year. These are a lot of fun. I joined Vivup just in time to go along to the 2022 Christmas party in Newcastle, and we’ve just had the 2023 one in Liverpool.

Oomph! Wellness

I was waiting for the right time to blog about my latest job. As it has now come to an end, I guess that’s overdue! So let me tell you about where I spent the past ~8 months: Oomph! Wellness.

They’re a company providing training and content to care homes in the UK. Activities and exercises for old folks! They’re not a tech company, or at least not until recently. For many years they’ve been doing face-to-face service provision, with teams of people going out to care homes to train staff in how to run activities and exercises. Another service involves sending bus drivers to take residents out on trips.

And then there’s a third service “Oomph on-demand” providing activities and exercise related content on the website by subscription. This side of the business has been expanding rapidly, particularly during the pandemic lock-downs. By “content” I mean videos, or just written instructions/inspiration on activities. Also things like puzzles. There’s quite a variety, and they’re constantly building it into an extensive library. The nice thing is that they’re drawing upon all their experience of working in care homes, and it really shows. That kind of grounding in the real world is something that is so often missing in tech up-starts, and of course their past work gave them a foot in the door to sell this web-based service to many existing clients.

So it’s a low-tech company transitioning into a subscription web content company. In fact they’d made that transition and were already selling on-demand subscriptions on a site built within wordpress. That was done by an agency with many wordpress plugins and not much coding. It’s quite impressive how far you can push that approach, but the agency had to say “no” to some of their newer ideas. They were going to need coding, actual data-modelling, and development. They were going to need me!

Obviously looking at a wordpress set-up with a tangled mass of plugins, I was quick to tell them this whole thing was going to need re-building (Ruby On Rails was my suggestion!), but I’ve done a fair bit of wordpress tinkering over the years so I wasn’t afraid to roll up my sleeves and implement some of their feature ideas in PHP.

I added a surveying and reporting feature, which had some enjoyable data aspects to it. I’ve had fun with c3js graphs in the past, so brought that in.

That whole thing turned into quite a rich interface. I probably should’ve taken the chance to learn a rich javascript framework. I did learn the way of wordpress a little bit. For example wordpress developers have a particular way of doing database querying, and AJAX requests. I also learned more than I wanted to know about how to persuade IE to print graphs correctly! Maybe I should blog some more about these learnings.

I was doing these things to meet the clamouring needs of some existing customers, to make the sale to a large new customer, and to lay the foundations for a grant funded project. I gradually settled into a role which can best be described as “the one-man tech team”. I quite enjoyed the challenge of always designing solutions and solving problems single-handedly, and presenting the solutions and options to a less-techy rest of the organisation, but I did miss having fellow developers to bounce ideas off. I was always a little out of my comfort zone in that regard. Enjoyable challenge but… I’m looking forward to re-joining a developer team for my next job I think (although I’m keeping an open mind)

While juggling php features and bugs, I started to look at doing a big website rebuild project for them, and I had got as far as architecting this as a Ruby on Rails project. I planned out how I could do the work single-handedly, however this was going to be lot to take on. The alternative for them, was to bring in an agency to do the rebuild, and I think it was sensible for them to go with that approach in the end. A better way of getting it done in time to meet some grant deliverables, as well as to do some Salesforce work which I certainly didn’t want to go near! It makes a lot of sense, and I wish them well with it. They’re a nice bunch of people and they’re doing good things. So… if you run activities in an old folks home (not sure that’s my core blog readership, but you never know!) be sure to check out Oomph Wellness!

It also makes a lot of sense for me to move onto something different. It’s been an interesting time, but as I look for a new job I’ll probably look to get back into ruby, probably within a larger tech team. That’s more my comfort zone. I have a few other ideas about what I’m looking for, but I’m also pretty flexible and keeping an open mind.

Linear time calculator

I keep doing the same calculation over and over again, so I decided to make an online tool specially for it:

>>> Linear time calculator <<<

It’s a simple enough calculation to work out a time estimate for something based on progress so far (assuming progress has a number). Could be used for anything I suppose, but I keep doing this for long-running data processing scripts, where a count is being spat out to a log file e.g. a count of records processed.

Here’s how the tool looks for script I was running recently.

Alternatives

Maybe I should’ve just worked out how to do the time format conversion and arithmetic properly in a spreadsheet, but why use a spreadsheet when you can build a whole dedicated tool for the job?!

There are some similar existing online calculators. You can do the time between two date times on timeanddate.com which I suppose is the error prone part.

Various online calculators do a speed calculation (division!) If you imagine “miles per hour” is “records per hour” it works.

Open Data Camp

I went to Open Data Camp 7 last weekend.

It’s an event I’ve been hoping to get along to for years now. Always lots of interesting looking twitter chat coming out of it. But this year it happened in London, and I managed to be on the ball getting a (free) ticket before they sold out.

I hadn’t particularly realised it’s the “unconference” format. It reminded me of the London “wherecamp” events I was involved in once upon time, although OpenDataCamp is even more unconferency. I mean the sessions were all very conversational. Didn’t see anyone with prepared slides, and often heard less from the session leader than from other people chiming in.

It was also different from wherecamp because there was a “pitching” session at the beginning to establish all the ideas. Those were then written up as a very busy schedule with four/five parallel tracks! Here’s all the sessions listed. In fact that has links on to more notes on each of the sessions too. Here’s the session choices I made:

  • “Bad data” – Discussing the precision of library visitor stats. I gave the example of bad data in company registries!
  • “Quantifying value of open data” – How to measure/justify the value of open data by tracking usage. The great unsolvable problem.
  • “Sorting out configuration so we can do cool things” – Really boiling down to: software can be hard to install and get working. Another great unsolvable problem.
  • “SPARQL 101” – A very quick intro to linked data querying. SPARQL playground and epimorphics qlassroom.
  • “What is non-commercial?” – Or does non-commercial really count as Open Data, or what other ways are there to avoid giving away all your value? As applied to the NHS!
  • “Open Procurement data for less carbon” – Influencing contract and procurement processes to (be more open and) reduce carbon emissions
  • “Joy Diversion” – An Open Data Manchester idea which I was always curious about. Walk round the block looking out for joyful serdipidousness. Quite fun. Quite similar to OpenStreetMap mapping (not as good!)
  • “OpenStreetMap” – My Session! I tried to introduce OpenStreetMap editing and using. Mentioning tags quickly led to questions and conversational diversions (as documented in the session notes) That made for an interesting different way of presenting it!
  • “Learning from our data mistakes / Open data horror stories” – People confessing failed attempts to open data, or opening data leading to other problems.

Some of the discussions gave me a strong sense of deja vu. We’re still collectively bashing our head against some problems of open data which feel unsolvable (not that I didn’t enjoy discussing them anyway!).

I also enjoyed bringing some OpenStreetMap (hobby) and OpenCorporates (day job) perspectives to discussions. I just realised I have over a decade of open data experience in both hobby/volunteering with OpenStreetMap and in my career spanning CloudMade, TransportAPI and OpenCorporates now, but I don’t always think of these things in abstract open data terms. My overriding impression of Open Data Camp was an enjoyable feeling of being among open data people again!

Yak Shaving and Woolly Jumpers

116_1269
Photo by Glen MacLarty – A woolly jumper being knitted (…probably not from Yak’s wool)

Do all developers know the phrase “yak shaving”? Apparently not. I see it being introduced to a new generation of developers here.

What does it mean?

“yak shaving” …the alternative explanation.

You’re an engineer. You have a problem:  you’re feeling a bit cold.

  • You turn up the heating, but you decide it’s better to find a jumper to put on.
  • You find a nylon sweater, but you want to do this properly. You need a woolly jumper.
  • You go out to the shops to buy a woolly jumper. There’s good options, but nothing in a perfect size and shape for you. You decide the only really kick-ass solution would be to knit yourself a good thick woolly jumper.
  • You look through some wool you already have, but since you’ll be spending quite some time on this knitting project, you decide you want some really premium quality wool.
  • Back to the shops to select the very best wool, but you’re not satisfied. If it’s worth finding good wool, you decide it’s worth going to the source of the supply chain where the wool is produced.
  • You go on a multi-day trek around the top wool-producing sheep farms, but as you learn more about wool you discover that the very best type of wool is from the Himalayan yak.
  • …and so it is that you end up on a hillside in the Himalayas, in the baking hot sun   …shaving a yak.

Origins

I like this explanation, but it is “the alternative explanation”, because I’ve no idea where it came from! Can’t find any reference to it on the internet. I think I was taught this by Andy Allan while I was working at CloudMade. A few years ago I introduced “yak shaving” to workmates at TransportAPI, to much hilarity.

For the real origins of the “yak shaving” phrase, all googles lead to Seth Godin‘s blog, which references, Joi Ito’s blog, which references an O’Reilly Book on productivity, and [oopse. now that’s going a bit recursive], but  no there’s lots of pointers to the real origin, which I find a bit disappointing: A “Yak Shaving Day” segment of a Renn & Stimpy episode.

I suppose it’s disappointing because Ren & Stimpy is so unbearably bizarre I can’t get my head around it, but also I was hoping for the origins to have some better link with our tech community use of the “yak shaving” phrase. Oh well.

New job at OpenCorporates

A month ago I started a new job. This is a big deal for me because I’d been in at TransportAPI for eight years. But time for a new job in 2018! I’ve just started at OpenCorporates.

OpenCorporates provides company listings data. It has a database of corporate entities, rather like Companies House in the UK, but bringing together this and similar datasets from countries all around the world.

Now “company listings data” may sound rather dull, but there’s a couple of aspects of this which I think are quite exciting: open data and… fighting crime and injustice!

Open Data

At one stage the Companies House website was only open during working hours (Yes. Hilariously the website’s clunky querying interface stopped working after 6pm!). Aside from making itself a target of general ridicule from tech people, companies house also became a target of the emerging “open data” campaign at the time. About a decade ago the guardian “Free Our Data” column was leading the march, campaigning for data releases. I also remember attending a “National Hack the Government” event hosted by the guardian, with all sorts of hijinks of civic hacking tech people who were producing better versions of government websites and, if necessary, taking data by force (brute force scraping) to do so.

From these kinds of mischievous beginnings OpenCorporates was born. I remember bumping into Chris Taggart at various conferences and events around this time, while I was generally running around beating the OpenStreetMap drum. Later I ended up working with TransportAPI, and TransportAPI ended up sharing an office space with OpenCorporates at the Open Data Institute for a while.

Companies House went on to get their act together considerably (spurred on by those early civic hackers no doubt). Meanwhile OpenCorporates continues to campaign for open data releases from other governments around the world. My point is, OpenCorporates is very much historically and currently part of the open data movement. I regard any open data work as a kind of “social good”, but if you think about it, company listings are really a flagship example of the kind of dataset any government absolutely should be making public and easy to use.

Fighting crime and injustice!

This brings me onto the other aspect. A more dramatic kind of “social good”. OpenCorporates, by making company listings data more available, is fighting crime and injustice …if a little indirectly.

A “registered company” might be a high-street shop or some other real-world bricks-and-mortar thing, but often not. Registered companies are legal and financial constructs which get used and abused in all sorts of ways. We see examples of dodgy dealings on any episode of “rogue traders” where it invariably reveals how bad people closed down a company, and re-started under a new name to shake off the authorities. It’s clearly too easy to get away with. Company listings data provides a little bit of an antidote by helping with transparency and knowing who you are doing business with.

But that’s just small time fraud. When I started at OpenCorporates they gave me a copy of the book “Treasure Islands: Tax Havens and the Men who Stole the World” by Nicholas Shaxson. It describes tax avoidance, tax evasion, money-laundering, and flows of criminal funds enabled by offshore “secrecy jurisdictions”. It builds up a picture of this as not just a side-show by a dishonest minority, but widespread deeply embedded system of corrupting influence benefitting the rich and powerful, at the expense of everyone else. The problem feels unsolvable as we realise that the corruption runs deep within all of our governments.

“Rich governments cannot be trusted to do the right thing on tax havens and transparency. Many will demand more transparency and international cooperation even as they work to frustrate both. They will call for reasoned debate as they engage in character assassination  secret deals and worse. They will talk the language of democracy and freedom the better to defend unaccountable, irresponsible power and privilege.”

It’s a pretty eye-opening and pretty horrifying book!

The book details some specific financial tricks and loopholes, but generally the idea for a global company (or crime syndicate, or just a wealthy individual) is to have your accountants and lawyers create a tangled web of different corporate entities registered in different jurisdictions, with maximum secrecy making it almost impossible for tax authorities or financial regulators to pin down your assets, profits, risks, and capital. There are many layers of financial trickery, but it’s clear that corporate registrations are the key vehicle for this kind of skullduggery.

As such, data on registered corporate entities is vital. It doesn’t solve all the problems of financial secrecy. Investigators will need to piece together a lot more about companies, their assets, the true owners and beneficiaries, and their links with other entities, in order to cut through the information void created deliberately to bamboozle them. But it’s a start. Company listings form a kind of information bed-rock upon which other information and investigation can be built. Whether it’s an investigative journalist looking to crack open a high profile case, or a tax authority, or a just somebody looking to establish the trustworthiness of a company, OpenCorporates helps enable this important work. It puts the information out there more publicly and in more useful formats to help shed a little bit of light on this murky world of crime and tax dodging.

Maybe “fighting crime and injustice”, is a super-style exaggeration. Folks like the Tax Justice Network are tackling this big problem more head on, but OpenCorporates helps chip away at the problem with a more bottom-up data-oriented approach.

The new job

So I’m getting my head into this new subject area, and joining OpenCorporates as a software engineer. It’s primarily Ruby on Rails development, but with lots of infrastructure and data processing challenges. Fun stuff! We’re based in the main pointy tower at Canary Wharf. We just had our office warming party in a new place 42 storeys up.

If you fancy joining me there, we are hiring back-end engineers, infrastructure engineers, and data analysts. Check out jobs.opencorporates.com!

Printable Calendar tool

Here’s a little thing I just found (I have lots of silly bits of code like this kicking around) :

Printable Calendar

Generate a calendar in a strip layout, suitable for printing, with lots of space for writing notes next to each date.

I originally did this as a visual basic word macro, then later as this php script.

I used to use this to do big calendar printouts to help with family logistics, usually at this time of year, covering the months around Christmas. That’s back when we four kids were all home for Christmas in Yorkshire, all with our own overlapping plans. Nowadays we might do a shared google calendar instead, but I remember the scrawlings all over the calendar wall used to be quite fun.

A tech manifesto from 2007

I’ve just been tidying some old content on this website, which I’d written just over 10 years ago now. Back then I wrote a kind of tech manifesto, or at least a collection of various broad thoughts of tech and the IT industry I was working in at the time. I haven’t worked as an enterprise integration consultant since 2009, and some of it is out-of-date in various other ways. Some of it feels like it’s noticeably coming from my naive younger self.

  • I hate the word “geek”. I don’t really hate the word of course, and that’s not the point I was trying to make. I stand by the idea that we should always work to close the separation between “geeks” and “normal people”. The march of technological progress does this for us of course. If I think back to 2007, the internet was actually a lot less mainstream. Fewer people with broadband at home (including me!). Fewer people required to use the internet or even computers as part of their work. It used to be that “geeks” were people who knew how to use computers and were super savvy with the internet. Nowadays that’s everyone. My 2 year old son is already getting the hang of it! Nowadays I see an interesting push to get more people from more diverse backgrounds to learn to code.
  • It’s a people thing was a piece complaining that clients should discuss high level requirements rather than skipping ahead to designing a solution. This is an accepted anti-waterfall principle rolled into “agile” these days. Perhaps it goes without saying, …except it’s still a common mistake. I recall a few occasions since writing that, when I’ve worked on projects which jumped to discussing a technical solution before getting a high level view of problems we were solving.

    I also talked about user interface design. I think I had a bit of a bee in my bonnet about the project I was on at the time, but I do remember leaving that particular project with great satisfaction at having implemented some of the UI ideas despite initially having them shot down. Since then there has been a couple of times where I’ve found myself surprised by colleagues’ failure to see obvious UI improvements. It makes me wonder whether user interface design is a talent I’ve not really appreciated within myself. Maybe I should do more of that kind of work.

  • With IT project politics, I was talking in general terms, but really I was bearing my soul about some frustrations with my consulting job. Some of the assignments with Green Hat Consulting involved parachuting into a pretty hostile environment. When I quit that industry and went to work in a more fresh funky start-up I left the politics behind on the whole, but of course you never really escape that kind of thing completely. I guess the golden rule I still have to remind myself of would be: work with people you like (and if you don’t, leave, because life’s too short)
  • Maintainable Software. Maybe I had a fairly simplistic view of what makes good software back then, but I think I was just glossing over the details. Obviously there’s a whole universe of coding best practices which make code maintainable, beyond “comments and meaningful variable names”. In fact comments are bad …sometimes. I think keeping up with recommendations and knowing which wisdom to follow and which to discard may be the real skill. Being an “opinionated coder”, and taking pride in your craft. In any case, I’m sure I was correct in saying that most developers consider their own code to be good code. …and I still didn’t learn to drive!

    I was also ranting about documentation. Again I think this was a bee I had my bonnet related a particular request to document a particular project at the time. But it remains a reasonable point, that documentation can be seen as an afterthought; a project delivery box to tick. I quite like documentation. The interesting challenge of trying to distill the most important hand-over information for a project, without making something which is just too long for anyone to bother reading. Also mechanisms to help ensure docs are kept up-to-date e.g. keeping docs close to the code or part of the code. I like that github have established (or helped reinforce) a convention of supplying a README file for each repo.

On the whole my “tech manifesto” of a decade ago wasn’t too bad, but a lot’s happened in 10 years, and some of these thoughts were starting to feel quite old. So I decided today that they belong in the blog archive.

But what would I write about if I were to pick some points to make about tech and the IT industry in 2017? (Not sure we even call it the “IT industry” so much these days). I don’t think I would try. Clearly such things are destined to go out of date. A single page of thoughts also feels incomplete, but maybe I should add some more deep thinking to the blog category ‘technology’

Open House rails developer

[Update: They’ve now hired a rails developer. Thanks for the responses! Still opportunities to help I’m sure. I will to try post an update on how to get involved]

I wanted to help promote this Ruby on Rails developer job at OpenCity which I think will be a really interesting thing for somebody, maybe a junior rails developer. Know anyone?

If you’re a londoner you’ve probably heard of “Open House” an event taking place each year where, for one day, you can take a look around lots of interesting buildings, for free, which are often closed to the public the rest of the time.

The Open House event is put together by a not-for-profit organisation called OpenCity, in their office near Aldgate. From there they organise this annual architectural bonanaza, coordinating hundreds of people (volunteers and building owners) with lots of careful planning …and some IT challenges.

opencity-diagram-currentopencity-diagram-1st-stage-rails-app opencity-diagram-big-rails-app

Being an interesting and fun not-for-profit organisation, I have ended up volunteering a bit of my time with them. Me and another volunteer have been planning a rebuild of a database system they’re using internally, which will later ripple up to some cool improvements on their public facing website.

To me this looks like a juicy challenge and a fun organisation to be helping out, so I’m hoping we can find a suitable available developer who feels the same! I’m not available myself, except in my spare time. I plan to be dipping in on this project from time-to-time, so I would be working with this person a little bit.

The project planning is at a very early stage, but I’ve been drawing the above diagrams which show… (left-to-right)

  • Their current set-up
  • An initial milestone introducing a rails app for their buildings database
  • And a final situation with websites and database consolidated

Hopefully in the end we will have managed to consolidate things and four different websites (or four different user journeys/permission levels within the same website) will be served by a lovely new rails app!

OpenCity are very open to our ideas, and one thing we’ve suggested, is to do this whole thing open source on github, as a way of being open to contribution from any other developers who fancy helping this organisation, but this is a pretty big job which will need somebody on it full time. Hopefully quite an interesting challenge for somebody! Please pass this on to anyone who might be interested.