Government Social Research
Trying It Out - The Role of 'Pilots' in Policy-Making Report of a Review of Government Pilots (PDF)
In Chapter 2, we listed some 27 recommendations arising out of the multipronged survey we conducted, the literature search and our deliberations.
They add up to a strong endorsement of the case for piloting new policy initiatives wherever practicable. And they provide enthusiastic support for the fact that the practice is now being embraced so widely across government.
There is no doubt that, costly and timeconsuming as some pilots are, the overall benefits they provide to good governance far outweigh their disadvantages. Naturally, they fulfil an important defensive role in guarding against the inclusion of embarrassing, often expensive and preventable mistakes into new policy initiatives. But they play a highly constructive role in promoting innovation (via explicit, small-scale experiments and trials), and in helping to fine-tune policies and their delivery mechanisms in advance of their national roll-out. In short, policy pilots have become an indispensable tool of modern government.
A large part of this report deals with the sorts of practical considerations that either enhance or diminish the optimal use of policy piloting in Britain. In sum they suggest that, excellent though some practice already is, there is still a long way to go before this will be uniformly true across all administrations, departments or, for that matter, across all pilots within any department. A great deal of practice still falls far short of its potential, and by no means all the obstacles to good practice will be simple to surmount. Some, such as the deep-seated suspicion in some quarters of RCTs, even in circumstances when they would seem to be an ideal mechanism, will take time to overcome, but surely will be. Others, such as the routine assumption that any new policy initiative must necessarily be introduced at the earliest possible moment, even when a small delay will help to ensure it is wellhoned, will probably take more of a culture change to rectify.
On the other hand, British policy pilots have been gaining in sophistication in recent years, both in their methodology and in their analysis, and many debilitating notions of what used to be considered possible or desirable have demonstrably been dispersed. We were particularly taken with the enthusiasm we encountered both among Ministers and senior civil servants who had experienced recent pilots in action. They had generally been convinced not only of the immense value of piloting in general, but – perhaps more importantly – of the desirability of more experimentation within policy pilots, designed explicitly to try out different models to achieve particular ends.
Britain still has lessons to learn from abroad, particularly about the methodology of piloting and its role in overall evaluation strategies. While our political and legislative frameworks remain less conducive to an optimal use of policy piloting than in, say, the US, great strides have been made in the past few years in both these respects.
Inconsistency remains a problem, as does a reluctance to embrace the best methods in all circumstances. Prior experimentation, trial and error, and the need for transparency all still need to be accorded their due importance in policy formulation.
We hope that this report will help to provide direction and momentum to a process that is already well under way.