Liveblogged notes from the Data Journalism: Mapping the future? event held by the Media Society last night. Please note: not a complete transcription, and captured in real-time and thus liable to inaccuracy, error, incomprehension and howling grammatical mistakes.
Chair: Raymond Snoddy. (Former Media Editor, The Times)
David Ottewell - Head of Data Journalism, Trinity Mirror
Martin Stabe - Head of Interactive News, The Financial Times
Jacqui Taylor - CEO, Flyingbinary Limited
It's not, in of itself, new. It's become a buzzword, but journalists have always dealt with numbers. One of the first stories of the Daily Illustrated Mirror was data journalism - about the sales of the paper.
There's prejudice between journalists and data journalists on both sides - we're not a different species. Data is another source - and we have new tools to deal with it. Data journalism needs to be aware of what it's good at - it's better at "what" is happening than "why" it's happening.
We're working at a different scale from the computer-assisted reporting days - and we have more tools to analyse and visualise the data.
There's a tendency to fetishise data. Data is just documents. It's a record of some form of transaction. Previously, this was reams and reams of paper - now it's digital. Many of these transactions are no longer on paper at all.
The only way to deal with these volumes is to deal with them in electronic form. Data journalism is the way journalism will be - whether we like it or not - so we might as well learn how to do it now.
We're seeing quantification of jobs - businesses that were based on instinct and intuition are now based on quantifiable analytics. Journalists are starting to be intellectually outgunned by the quants of other industries. If we can't do our own, independent analysis, then we're at the mercy of the PR machine.
Placement of stories now has to be justified against analytics data - but this is trivial compared to other industries. The old tastemaker roles are being replaced by quants. There's an imperative to have data to justify every decision you make. And the computing power to do that is available to everyone.
There is a problem with the "filter bubble" - ending up locked into pre-determined conditions by algorithms. But that's distraction. The data is out there - if we don't use it, others will do.
The skills gap in newsrooms is a big problem - having to explain the trail to editors who don't understand is hard. You need to be able to share your work - to justify it, right back to the raw source.
Aren't we talking about getting the stories out to an audience? Based on the way the web is going, that won't be an mainly English-speaking audience in a few years. Gen Y will soon be in charge of the workforce. The Millennials have a very different way of approaching this discussion - they're a visual generation. Are we going to see this as a challenge or to trailblaze for this generation?
You can look at how people are exploring the data in an interactive tool, and find other stories in that. You get more stories than journalists can find alone. For example, what happens in Tokyo is very different to what happens in Kansas - based on the clicks we see. Maybe we could run different stories in different parts of the world.
David: One good example of a story that we couldn't have done without data journalism is the Government's instance that councils register every expenditure over £500. It's incredibly difficult to analyse that level of information. The detail was so vague as to be not useful. So, instead, we just searched for interesting payments, things that were unusual. So we looked for the biggest unique payments - one offs to companies. When we did this to Manchester Council we discovered they'd paid £100,000s to the Australian Swimming team. We investigated and discovered they were paying them for training - and the team said that they'd have done it for free!
The council cuts is a huge agenda. The Northern urban councils- they've lost a lot more money - and you can map that, and then ask the politicians why.
Martin: This is social science on deadline. We're taking the best practices coming out of academia and applying them to journalism. Chris did good work on GSCE results - he got the database exclusively through negotiation. He was able to milk that for weeks and month, finding story after story. Other journalists thought he was making it up even though he posted on his blog how he did it all.
Jacqui: Do you see this as a challenge or opportunity? Gen Y are more comfortable about learning a bit of code to pull this all together. Do we want the English journalism industry to disappear when the web is no longer predominately English language? What opportunities are there for revenue when people spend between eigth to 10 minutes with an interactive?
David: I'm not sure that these are skills that every journalist needs. Would anyone want to read a newspaper that was just data journalism? I don't think so. Our challenge is to make sure we're coming up with stories that are worthy of the front page. If you don't have an awareness of data journalism skills, you're missing a important skill, though.
Assorted hackademics in the room, including Jon Hewett, suggest that literacy rather than skills are important to all journalists. The Interactive Journalism MA students at City University(disclosure: I teach on that course) are going into jobs on nationals. People are advertising for data journalism skills.
David: This is unambiguously one of the areas that there are jobs in journalism.
Martin: The freedom of information act was the trigger for this - but extraordinary rendition was the first big data journalism story that was done. If you get a big leak now, it's going to come on a big hard drive. If you don't know how to search that in a meaningful way, you can't do the big investigations any more.
Liz Hannaford: With the exception of Jonathan Hewett's course, I don't see journalism schools jumping on this. The young journalists with these skills are finding their own skills and the making their way into journalism.
Jacqui: Web science is a multi-disciplinary science that was only invented four years ago - Our PhDs were co-created with other universities and universities.
Martin: The single most useful course I took was statistics for social scientists. Journalism schools don't get it (with the exception of City). I'm quite pessimistic about getting these skills from traditional routes. The NYT picked a PhD-level statistician and turned them into a journalist to get the skills they need.
David: My heart soars when I get an application from the City Intearctive course.
Martin: What used to be hard is now easy. The bar of doing intelligent clever stuff is rising all the time. If you're a data journalist in the US, you're doing really well, jumping from job to job, getting paid better and better. This is unheard of in journalism. The NYT is hiring famous coders. The bar of the talent to do cutting edge stuff is really high.The top end of this game is well above just spreadsheets.
David: Coders skills are at a premium - and publishers don't realise this.
Martin: All the stuff we're interested in is propriety data - not government stuff. We don't have access to it without paying, and we can't often use it as we want. Open Data is only what the Government wants you to see. You also need the stick of FOI. Germany is the hotbed of data journalism in Europe - but they haven't had effective FOI until recently.
Jacqui: Post-Snowden we can't look at the social web in the same way. The Gen Z cohort already use the social web differently anyway. The social monitoring platforms went dark post-showdown. It's created a trust issue. A line might have been crossed by the way Governments are behaving.