Published daily by the Lowy Institute

How do we know if counter-radicalisation programs are working?

How do we know if counter-radicalisation programs are working?
Published 16 Jun 2015 

The Federal Government recently announced it will allocate $64 million to 'measures to counter radicalization and violent extremism'. Yet, as Hussain Nadim has pointed out on The Interpreter, the Government is spending the money on programs without any examination of whether they will actually work.

If we were talking about a new cancer treatment, we would find this state of affairs shocking. Yet Australia is prepared to spend money on deradicalisation without any indication as to whether this money will reduce the risk of terrorism.

Nadim has some good ideas about how we can improve on current deradicalisation efforts. But testing whether they work or not isn't easy. Let's say we try out a deradicalisation program on three young Pakistani men like the ones Nadim interviewed. Suppose they all turn away from terrorism after they complete the program. Great. But does that mean the program worked, or would they have turned away from terrorism anyway?

Perhaps we could compare them to three otherwise similar young men who were not exposed to the program. That's better, but we're still talking about just six people. Maybe the result has got something to do with their individual life stories and the apparent success of the program is just luck. OK, so let's look at a larger number of people. Now we're getting into the world of the randomised controlled trials, or RCTs. [fold]

RCTs are standard in medicine. If you've been sick in Australia at any time since 1998, chances are the treatment you got has been tested through an RCT under the oversight of the Medical Services Advisory Commission. Applying RCTs to public policy is a more novel proposition for many, but they are increasingly becoming standard in many areas of domestic politics. Many US states require new education and welfare programs to be tested through RCTs, while the UK's Behavioral Policy Unit is applying them to a range of policies at the national level there. In an RCT, one assigns individuals at random either to receive the treatment or not (so as to ensure that the end result can't be due to any other major systematic factor), and does so on a large number of subjects (to ensure that the results aren't driven by the idiosyncrasies of one or two people).

I'm by no means the first person to advocate using RCTs to gauge the effectiveness of counter-terrorism policies. Writing in the New York Times, Anthony Biglan approvingly cites the work of Chris Blattman, an economist at Northwestern University, who used an RCT to show that cognitive behavioral therapy proved effective in turning young men away from joining rebel groups in Liberia. Biglan launched a passionate appeal for similar work on radicalisation among Muslim communities in the West.

But the lack of RCTs on counter-terrorism isn't necessarily a matter of mindless resistance to new methods. Applying RCTs to a subject like this is far trickier than in many other areas of public policy.

First, the behavioral outcome we're looking to change is rare. The vast majority of young Australian Muslims do not engage in terrorism or jet off to join ISIS. This complicates estimating the effects of any programs. Second, the very act of recording individual participants' behaviour may change that behavior – individuals who know they're being tracked by a team of investigators are less likely to join terrorist organisations regardless of whether a de-radicalisation program has had any effect on them or not.

If we were to run an RCT on deradicalisation, we would therefore best be advised to look at the effect on attitudes rather than behaviour. Even here, though, there are problems. You might get a young Australian Muslim to admit to harbouring pro-ISIS sympathies in front of a white researcher. But when you consider the risks they are running of falling foul of the authorities, it's quite likely they'll do their best to keep the views we're most interested in studying carefully under wraps. Fortunately, however, researchers have developed a number of ways of getting individuals to express views which, for whatever reason, they would rather not admit to holding. So-called list or endorsement experiments have been tried out, for instance, in conflict zones such as Afghanistan, with interesting results.

These are simply preliminary sketches of how we could begin to measure what young Australian Muslims really think of ISIS and its worldview. Once we can do that, we can begin to apply RCTs to analyse deradicalisation programs the way we do with medical treatments. And hopefully with the same success.

You may also be interested in