Paul Bradshaw - Data Journalism Heist

Every good heist begins with a recce: a reconnaissance mission to check out the site of our operation. Data journalism has a number of key sites to recce:

• If your government, region or city has an open data portal, that should have regular updates. You can find a list at CKAN: open data sites range from the UKs data.gov.uk, to regional and city sites like Waterloo in Canada, Emilia-Romagna in Italy, or Chicago in the US;
• If theres an office of national statistics, sign up for updates. One list of national statistical bodies can be found on Wikipedia;
• FOI requests are often a good source of data. If you have a site in your country that allows people to send and monitor these (such as Whatdotheyknow.com in the UK and AskTheEU for EU-related FOI requests) then these often provide an alert facility. Also look for specific bodies disclosure logs - where they publish FOI requests received.

Take some time to check these out. Its a good idea to have data regularly come to you - either by email or, if you use an RSS reader to follow feeds from various sites, that.

If the website has neither email nor RSS updates, try using ChangeDetection.com - this will send you an email when a webpage changes.

If you are reporting on - or particularly interested in - a specific field like crime, health, education, welfare or the environment, try to find bodies (local, national and international) that publish data regularly in that area.

Here are just a few general sources of regular data in the UK alone:

• Education: the Higher Education Statistics Agency (HESA), the Higher Education Funding Council (HEFCE) and Universities and Colleges Admission Service (UCAS) all hold data on universities and students. Ofsted, the regulator of schools, hold data on pupils and education at pre-16 level.
• The NHS Information Centre (NHSIC) and Health Episode Statistics (HES) hold data on hospital admissions and local doctors.
• NOMIS holds data on the labour market: where people are employed and unemployed.
• Data.police.uk holds data on crime and policing

This scouting mission concerns a typical type of data which you might find landing in your alerts regularly: local government spending.

In England every local council is required to publish data on its spending over 500. To find it, search for expenditure 500 and the name of a local authority. In our case, were going to look at Birmingham - the biggest local authority in Europe.

What the data contains doesnt really matter here: the point is the exercises youll be going through in looking at it. You can apply the same process to most regular public datasets, whether its employment data, environmental information, or weather.

Birmingham City Councils expenditure data is at Birmingham.gov.uk/payment-data. Youll find monthly spending data going back over the last year, and can request older data by email.

It comes in two formats: PDFs, and spreadsheets. Given the choice, always avoid PDFs.

If you prefer to work on some international data, download loans data from the European Investment Bank - change the search to the widest possible criteria and then use the (easy to miss) Export link at the bottom of the page to get an Excel spreadsheet.

The spreadsheets are shown here as a Microsoft Excel icon, but they are actually CSV files which will work on any spreadsheet software, including free options like Google Drive spreadsheets and Open Office (which can also open Excel spreadsheets).

CSV stands for Comma Separated Values - this means that the value in each column is separated by a comma like so: Name, Date of birth, Address. When the spreadsheet software opens this, it replaces each comma with a new column.

Click on the most recent spreadsheet version of the spending data (not the PDF) and download it to your computer. Make sure you save it somewhere you can find later, like your desktop.

Once downloaded, open it in your preferred spreadsheet software - either by double-clicking it or uploading to a web-based tool like Google Drive.

Now, were in.