Updates


18
May 12

Working the internal data universe

When I started first writing code, I wrote in a typically haphazard way. (actually I still do this, but now have half an eye on well formed code style)

But what I definitely used to do was think about how to store persistent data that the application needed in a relational database backend. I wrote a CMS from scratch to host this website on (long since decommissioned), and a system for working out how much parents of nursery children owed based on the time their children had been in the nursery compared to contracted time (there were penalties for late collection). Both of these web apps required persistant data in the backend, and at the time my first recourse to a data store was a relational database.

Working with a relational database didn’t sit well with my style of fast iteration to get a problem solved, then tidy up afterwards, to make sure that the code was workable and readable.

Having to constantly add columns to a table as new peices of information were required to be stored, for either optimisation or to support some new feature, was a pain. It broke code on more than one occasion, and created hard to pin down bugs that were usually the result of using a select statement that called for named columns that were now either renamed or moved or gone.

I wasn’t agreeing my schema up front, because I didn’t know exactly what data I would need to solve the problems. I admit I could have used some coding strategies that would have abstracted the database layer from the application layer, but even though I tried to implement MVC, it seemed to just create more code and dependancies.

Contrast that to how I think about solving a code problem today.

Graph thinking; graph databases; the graph; are all things I didn’t know about when I started writing code. Now, when I am thinking of solving a problem I think about exactly two things.

  • What data does my application use?
  • What do I need to show to the user?

The answer to the first question is not a database schema, nor is it something that will break my code if changed. It is a description of the data my app will interact with, but described as a graph so that I can just add more description as appropriate.

The answer to the second question, is that each bit of code knows about a small part of the graph that it needs in order to function correctly. Each function will know exactly which bits of the graph it needs, and how to get it. If the shape of the graph changes it will either flag an error if something critical is missing, or simply ignore the changes and continue working as before.

I said earlier that I didn’t agree my scheme up front because I didn’t know what data I would need to solve my problems. I realise now that if I had simply described the things my application would be interested in, I would have had more than enough data to get going with. I would also have been able to add new data back into the graph thus persisting the solutions to the problems I was solving.

But I couldn’t do that with a relational database because of the key difference between using relational databases and graph databases: I am no longer forced into using the assumption that the edge of my relational database is the edge of the world, and that anything outside of my database does not exist.

There are challenges to this way of working too, but I find it a far more natural way of working. I even find that new opportunities for data display or analysis become apparent as the data morphs into shape.

In short: working with graph data is more like working with the data universe that is inside my head. And that view of the world is exactly that, a view of the real world.


30
Mar 12

A letter from the Middle Ages

This post originally appeared on the Talis Consulting Blog

Well actually, not just one letter, but over a thousand letters from the middle ages.

Last weekend, the National Archives held a Hackathon in the reading room at Kew. Around 40 developers and interested people took data from the National Archives and played with it.  There were new mobile interfaces for the NRA discovery API; collections of tweets mined for the data and PDFs they contained; stats on historical participation in the olympics pulled from the archives and shown on interactive maps. In all it was a fun weekend with lots of smart people in the room and very quiet but rapid typing on keyboards to get something finished by the 4pm Sunday deadline.

Prizes were:

  • 1st – Jonathan Tweed and Kai En Ong (ably assisted by Michael Smethurst, Faith Mowbray and Paul Rissen). A hack that pulls out data surrounding people & places in documents tweeted by @ukwarcabinet (and which – for a hack – is beautifully presented!).
  • 2nd – Jamie Mahoney – Debtors & creditors dataset hack maps the most popular lenders & shows who’s borrowing from where – Show me the money.
  • 2nd – Tim Hodson – A hack showing who wrote to whom in the middle ages.
  • 3rd – Crystal and Steven Hirschorn – A hack showing participation in the Olympics on an interactive map.

You can read more about these entries on the National Archives blog.

I hope you’ll forgive my showing off of my joint second prize winning contribution to the pizza and jelly baby fuelled hack fest.

I took a suggestion from Paul Risson as a personal challenge, and started puling the data that I wanted into a new CSV file.  I then converted that CSV file to a rudimentary RDF based model of the letters and people that the data described.  I now had a graph dataset which captured – in the way only a graph can – the network of relationships between people who are corresponding. It was then a case of finding a suitable javascript library to render my graph as a visual and to allow people to find out about who wrote to whom without cluttering up the graph diagram.


24
Mar 12

Riding London

I find myself increasingly frustrated by the idea of having to use public transport especially when i have heavy bags to carry and know that i could do it much more easily by bike. So last night I spent extra effort to make sure that I could ride my bike across London today. My folder is a Birdy Blue, and as such has mudguards that are integral to the bikes stability when folded. They are also vulnerable and so have got somewhat damaged over the last two years. So much damage that the front is held together with black gaffa tape and the rear had split in half.

My new set of mudguards arrived last week, and so I would normally have spent my Saturday at the Wolverhampton bike shed fixing other people’s bikes, and then in our yard, fixing up my own. However this week I was bound for London to attend a hack day, so Saturday bike tinkering was out of the picture.

For a while I was evaluating the options for travel from Marylebone to Kew, and thought that I could probably do it via the underground fairly easily. But then I thought of the bag of stuff I’d be carrying and how I would have to lug it around the underground stations. The wheeled bag is a drag (literally) and doesn’t fit onto the front rack of a boris bike, and a backpack is out of the question because of the weight.

I kept looking at the Birdy and realised that I wasn’t going to be happy on this journey without it. Therefore I set to and after removing two stubbornly steel-to-aluminium bonded screws with the aid of a drill, I am now enjoying the prospect of cycling to Richmond on a gloriously sunny London day.

Cycling is my favourite transport.


6
Feb 12

Critical Mass, February 2012

If you are into cycling in any way, you might like the opportunity for a friendly cycle round Birmingham.  Birmingham has a bit of a car centric attitude toward town planning, so it is great to be able to cycle around the center en mass and be a very visible part of the evening traffic.  Oh and we drop in at a pub on the way home. What’s not to like?

Here is a short video of last weeks Critical Mass.

Critical Mass, Birmingham, February 2012 from Tim Hodson on Vimeo.

A Critical Mass ride around central Birmingham. It was cold, getting close to if not below zero, but still a good ride.

Meet at Pigeon Poo park (by St Philips, Temple Row) for an 18:30 start.


14
Jan 12

Ubuntu, Copying Partitions and UUIDs

After some rather unsettling moments when everything in RAM kept running, but the root filesystem quietly disappeared, I decided to clone the root partition onto a second drive and boot from that. On investigation it looked like the first hard drive – /dev/sda –  had been hitting a max temprature of over 130°C.

The thing that was really puzzling me was how to tell Grub 2 that it should use /dev/sdc6 as the new root partition. I followed several sets of instructions, but no matter what I did, it always chose /dev/sda6 as the root partition.

I then tried using the excellent boot-repair disk to see if that could do what I wanted.  I ticked the option I wanted which was to use /dev/sdc6 a the default boot partition.  I applied the changes and still it booted from the wrong partition.  boot-repair sends a clear and well thought-out report to paste.ubuntu.com, and while looking through this I happened to note that the cloned partition had the same UUID (Universally Unique IDentifier) as the original partition.

light bulb!

I figured that this must be causing the confusion, so a quick google pointed me to this post by Paul Goscicki, which confirmed that it was a likely cause.  So on running tune2fs -U random /dev/sdc6 , and then re-running boot-repair, I now have a system booting from the correct partition.

UUIDs are obviously not always UUIDs!


8
Dec 11

Kasabi.com’s Developer Advocate

Since the beginning of this week, I am officially now working for Kasabi.com.  My role here is to find ways to make the adoption of Kasabi.com’s suite of datasets and easy to consume APIs even easier for developers who need a quick way to get at the data that powers their apps.

You probably already know that the core data hosting and ready-provisioned APIs of Kasabi datasets allow you to get a data backed application out of the door much more quickly than you would if you had to incorporate data hosting and access infrastructure within your project.

We want to concentrate on helping you build something fantastic that is compelling for your user groups.  We’re going to be doing this in several ways; by improving our documentation; by holding webinars to demonstrate key features; by running hack days to give you space to play and learn how to effectively work with the Kasabi APIs in order to get the data you want.

This is an exciting time.  We are building something that feels like its moment has come.


8
Dec 11

Cycle safety: A longer amber phase for traffic lights

While reading the DfT’s publication of a report on Infrastructure and Cycling Safety, it occurred to me that increasing the length of the pre-green red/amber phase at traffic lights, and allowing cyclists to cross the junction in this phase could be a relatively cheap way to improve the visibility and reduce the risk of collision by cars.

What do you think?

What are the pros and cons?


17
Oct 11

Birmingham’s Grey Days

People rushing to appointments with desks,
Collars high to keep out wind,
Blown leaves that swirl and loiter,
Linger high,
Then tumble back to flagged floor,
Sticking in reflections that ripple as,
Feet tramp,
Splashing hems with liver spots.

Bright snatches of colour,
brave colour,
Valiant flag-poled scarves,
Disordered tresses,
Ruffling dresses,
Thick coats,
Woollen suits,
Patent boots,
Signalling Birmingham’s grey days.


19
Sep 11

Crunchy Mince and Cavolo Nero with Gnocchi

Over the weekend, I made the following dish. I only wish I had taken a picture of it as it was so delicious.  Here I record the recipe, because I know I am likely to want to do it again sometime. As with the majority of my cooking, it is experimental in nature, so feel free to suggest a more interesting name or point out any similarity with world cuisine somewhere…

Description

Gnocchi in a white sauce, provides a soft backdrop to crispy beef mince and crispy green cavolo nero with a hint of oregano and carraway seed.

Ingredients

  • Some mince
  • Some carraway seed
  • Some oregano
  • Some gnocchi
  • Some cavolo nero – cut into thin strips across the stem
  • Some flour
  • Some butter (or margarine)
  • Some milk
  • Salt and pepper to taste

Method

In a sauté pan, melt a tiny bit of butter and then load up with the mince. Add the carraway seed and oregano while the meat is sizzling in its own juices. Let the mince cook through and make sure it is well separated (i.e. not cooked in big clumps). Add the strips of cavolo nero to the mince and cover. Keep an eye on the mince, you want it to start going crispy, but not burnt, and you want to keep the environment in the pan quite steamy so that the cavolo nero is part steamed, part sautéd; this will mean adding a few tablespoons of water on a regular basis.
While you are doing all that (and don’t you hate it when cookery books say that and you realise you should have started this bit half an hour ago…) make a white sauce. Melt butter in a saucepan, take it off the heat, mix in some flour until it is a smothe paste, then drip by drip add milk while stirring in to keep that smooth consistency. then pop it back on the heat and keep storing. You could add some grated cheese at this point, but I didn’t have any in the house.
Cook the gnocchi according to the instructions on the packet (yes I cheated).
Server the gnocchi with the white sauce poured over, and the cavolo nero plus mince to one side.

30
Aug 11

Digital open space

Just a quick thought… In this age of social networking via portable smartdevices, It would be really interesting for architects and planners to explore how their development of open space can become a part of online social space.

Some ideas for you…

  • have lots of easily identifiable public art sculptures that can act as meeting spaces.
  • number the squares in the square so that people can find a square stand in it and tell their friends to meet them at that square. could be the basis of some rather interesting games too…
  • street furniture that can be tweeted to provide identifiable meeting places
Any others?