John Ross, ‘Crowd research projects inevitably attract sabotage, and managers should judge whether to use citizen science on a case-by-case basis’ The Australian, 10 September 2014

These are the conclusions of a “first principles economic analysis”, which has found that every crowd-sourced project can expect at least one malicious attack.

The findings, published in the Journal of the Royal Society Interface, emerge amid unprecedented reliance on citizen science to reinforce anything from bird, bee and sea urchin counts to predicting disease trends and identifying distant galaxies.

Last week, a review found that volunteer data underpinned up to 77 per cent of studies into the effects of climate change on migratory birds. And today, a University of NSW-led team has issued a call for recreational sailors to become “citizen oceanographers”, testing remote waters from their yachts.

But the exponential growth of volunteer research and content development has sparked a sabotage boom. Wikipedia now has legions of online “bots” and “cyborgs” scouring its pages for vandalism. Open-source projects to map Middle East conflicts have been suspended because of fears of misinformation and infiltration, and crowd-funding projects have been hacked and discredited. In 2012, the winners of the $5000 “Tag Challenge” — an online competition to identify suspects in a simulated law enforcement exercise — were almost derailed by a Twitter smear campaign. The previous year, a campaign of attacks cost a University of California, San Diego team victory in the $50,000 “Shredder Challenge” to reassemble shredded documents.

At least one of the team’s 3600 volunteers sabotaged its efforts night after night, removing pieces from the online jigsaw and putting them on top of others.

The study finds that this sort of behaviour is the norm rather than the anomaly. And co-author Manuel Cebrian — who led the UCSD Shredder Challenge team — said the findings applied even when money was not at stake.

“Vandalism happens because there are always two parties — China and Japan; pro and anti-climate change. They’re going to compete for knowledge, money, you name it,” he says.

Dr Cebrian, now a computer scientist with National ICT Australia, said crowd-sourcing saboteurs had been considered outliers. “I was never convinced,” he said. “If it was an irrational act by lone wolves, they would never act in groups and they would not invest that much effort. Sabotage is a rational tactic by smart people who invest a lot of effort.”

The study applied game theory analysis to a simple form of crowd-sourcing — a competition between two parties, each of which can rope in volunteers. It found that it would always be in a weaker team’s interests to sabotage a stronger team.

Attempts to prevent sabotage would discourage the stronger team from reprisal attacks, but further encourage the weaker team. “This contradicts the intuition that making attacks more costly helps prevent them,” the study says.

Dr Cebrian said the results had surprised the team. “We thought that under certain scenarios, it wouldn’t make sense economically for people to attack. But we couldn’t find those scenarios. It always makes sense.”

He said the results held regardless of the number of participants and the nature of the activity, because there would always be people with ideological objections — either to the activity or to the volunteer ethos.

He said it had taken him just 10 minutes to find vandalism in a supposedly sabotage-free crowd-sourced project to digitise part of the National Library’s collection.

“The take-home message is that sabotage is here to stay in citizen science. When you design a system you should first (decide whether) you have enough technology, manpower or verification energy to deal with sabotage.

“If you don’t, maybe you shouldn’t do it.”