Nerd Box: How to make robots do your reporting work

Cover robots: Courtesy of Nick Evershed.

Nerd box, n.

What older journalists in the U.S. newsrooms where some of us came up call a little feature that runs alongside a data-heavy story to explain the methodology. It’s intended to reassure readers, who are generally well aware there’s no reason to trust journalists with numbers, that the authors of the analysis took a reasonable approach, checked their work (with experts if need be) and did not pull said numbers out of thin air. It’s what we’re calling a new occasional feature on the Walkley Magazine that will highlight the work of data journalists doing interesting things so others can learn from them.

Here’s Nick Evershed of Guardian Australia on his Twitterbot @AusDisclosure, which tells the world when Australian politicians update their required disclosures of gifts they’ve received.

Kate Golden, multimedia manager

By Nick Evershed

When political donations are declared by parties and politicians each year, there’s a flurry of stories from the media, and then interest wanes once the initial stories are out the door.

However, parties often make amendments to their declarations. Sometimes, this is well documented, such as when the NSW Liberal party declared a swag of old donations, or when the Victorian branch of the Labor party filed an amendment leading up to Bill Shorten’s appearance at the trade union royal commission.

Mostly these amendments go unnoticed, and unreported. As Crikey’s Bernard Keane says:

“Political parties routinely get away with declaring hundreds of thousands of dollars in donations years later, long after anyone has stopped scrutinising them.”

Politicians' *updates* to their disclosures don't get much attention.

Politicians’ *updates* to their disclosures don’t get much attention.

There’s a similar issue with the register of pecuniary interests. While it’s entirely within the rules to make ongoing updates to the register, which records politicians’ investments, memberships, gifts received, and other things, most media scrutiny happens in a single burst when politicians are sworn in, with little following.

And it’s understandable. Monitoring each party’s donation declarations for each year, and checking every politician’s register of interest declarations every day would be a laborious task.

Very laborious. At a recent gathering, Evershed challenged fellow journos to read this sample disclosure.

Very laborious. At a recent gathering, Evershed challenged fellow journos to read this sample disclosure.

The solution to this problem? Make a robot to do the work for you.

Automation in journalism isn’t a new concept, either. Programs have been written to automatically write a news story for the LA times every time there’s an earthquake, or to take a live feed of sports results and statistics and turn it into a match report.

For my project, I wrote a program in three parts. The first checks every politician’s register of interests, the second checks every party’s donation declarations on the AEC website going back to 1998. If either program finds an entry that has a newer date than what I have stored in the database, then it is marked as an updated entry.

Then, the results are published via a twitter bot, @AusDisclosure. Technically, it’s a glorified web scraper written in Python (a programming language) and a very simple Python library that makes it easy to access the Twitter API (an API is a protocol for interacting with another application).

In future, the disclosure bot might send out email updates, or cover the lobbyist’s register, or donation declarations from so-called associated entities. I’ve got a number of improvements in mind.

While I was initially hesitant to make something public that I could have used internally to get information few others would be aware of, publishing the results via social media is actually beneficial to finding any story. The more interesting updates are naturally discussed more and become more prominent, and people are able to see updates relevant to their local member that a national publication might not consider worth covering.

This isn’t the only area ripe for automation in Australian journalism, either. Weather coverage, financial market reports, labor force figures, sports – basically anything with data available that is regularly updated is suitable.

This isn’t going to replace real, human journalists any time soon as any stories produced with these methods will be necessarily short and lacking important context. A human journalist can then pick up the bot’s copy and improve it, or instead concentrate on tasks that a computer can’t do.

All of the code is available on my GitHub account.

Nick Evershed’s slides:

Nick Evershed is the data and interactives editor at Guardian Australia. @NickEvershed