A slightly belated write-up of the things I saw at Open Tech this year. Keywords data, visualisation, google refine, data, more data…..

QR codes with all you need to know about Open Tech

Didn’t stay as long this year, due to a timetable clash dilemma: sit through nine hours of interesting speakers and discussion, or follow the efforts of 15 men as they try to win the Heineken  Cup. Chose to see as much of Open Tech as possible, then rush home to follow the rugby. Glad I did – talks were fascinating, and Northampton confounded all the pundits to put in a storming first half. Unfortunately they ran out of moves in the second half, and Leinster took the cup, but I was so pleased they made the final then gave everyone something to talk about! Maybe next year.

Back to the talks. Started off in the Open Data Cities session, hearing about activities in Brighton & Hove, Manchester, and Sheffield. Thankyou Julian Tait, of Future Everything,  Greg Hadfield and Jag Gill for fascinating insights into how three different cities are approaching the open data question. Manchester’s activities are advanced – various groups have come together and the creation of Data GM has sparked off other initiatives, including Linked Manchester, which in turn encourages other developers to build stuff. The movement in Brighton & Hove started with CityCamp, when a range of activists got together. Their challenge is different to Manchester, as the city is much more coherent, for example working with just a single local authority. Sheffield is at an early stage. After initial levels of enthusiasm, the question now is how to sustain the community and meet a range of diverse interests. There was discussion around the range of things that can be done with open data – from apps, visualisations, transparency and analysis to engagement, and a valid comment that for people to achieve the latter, there need to be advances made in data literacy levels. Jag’s slides are worth taking a look at – not least for the innovative range of visualisations he used to illustrate his points.

The discussion which followed raised two interesting points: co-creation will help get people interested in the area they live and/or work in and how it functions; and the besides being interesting and useful to the citizen, authorities also should be able to make use of the aggregation of open data – and act on the intelligence it provides them.

Next session was in two parts. First up was a presentation on behalf of Sandbag whose aim is to expose the iniquities in the carbon credits market through visualisation. This is an extremely complex area, but they have loads of data, and next aim to illustrate how lobbying activities have got us to this position.

Next was Christopher Osborne who gave an extremely polished and entertaining presentation about visualising big data.  Besides showing some of the impressive visualisations made by Itoworld, he talked about the importance of stories which need to accompany and give context to data and give purpose to visualisations.  He talked through the example of Haiti and open streetmap (that visualisation is hypnotic and definitely worth a look), but it is the story that goes with it which really makes an impact.  He covered examples both ancient and modern – from Florence Nightingale and the mortality data of war, to recent work done to illustrate the impact of congestion charging.  This latter story really showed the benefits of conversation. The timeline was: collect data, analyse, come to some early conclusions, talk about them, realise they didn’t really illustrate the whole picture, talk about it some more, with people beyond the obvious community, and end up finding an explanation you would never have thought of based on the initial data.

Next session was in three parts – loosely introduced as two ways to solve data problems, followed by an update on the LinkedGov project.  The first talk was on Scraperwiki – the detail of which I confess was a little beyond my technical competence, but I can see the benefits – especially of commissioning many individual scrapers which combine to cover a lot of material. The speaker talked about the OpenCorporates project, which aims to find a unique identifier for every company in the world., and how the Alphagov project used it  to gather data from government websites. Confess I felt a slight twinge when hearing about the latter – while I can see now the overall benefits in what the Alphagov team have achieved, and look forward to working with them to make it even better, as a manager of a government website it does feel odd that our material was scraped and a product was created all without any contact with our team. Data without context can lead to some odd conclusions – far better that questions are asked up front – after all, the end result should be better sites now and later.

Second talk was on Google Refine -a product described as a spreadsheet on acid. Its a product developed from freebase. It enables you to take a huge spreadsheet and, using a technique similar to xcel facets, you can sort and refine data, spotting anomales and fixing them extremely quickly.  Additional point – its open source.

The last session I went to was given by Bill Thompson- a regular at Open Tech in the main hall. He posed the question: Can the hacker ethic make a better world.  In talking about the way the open data movement has evolved over the last 25 years,  he drew comparisons with Gutenberg and Ghengis Khan. Both led movements which brought about immense change – Gutenberg to the society of which he was a part, and Ghengis to the cultures which he and his hordes came into contact with. Data reuse and the changes that brings, and the power of the network IS already reshaping our world.  [I guess you had to be there to follow the analogy].  His last question was about what might replace the protestant work ethic – a society based on social justice and work done not completely for personal benefit?

At that point, I left to catch a train. A fitting session to end a day of diverse talks with the common theme: open data – now the concept is out there, what might come next?