Government Social Research
The information in this report captures some of the main points made by keynote speakers in the plenary sessions. It is designed as a reminder of the issues under debate for those who attended, and as a pointer for further discussion by those who did not. It is not, nor is it intended to be, a comprehensive record of all that was said. Although every effort has been made to capture the points made accurately, this information has been compiled from notes taken at the time by GSR members. It is not written by the speakers themselves, although they have seen and approved the summary. Any views expressed are not necessarily those of GSR. Not all speakers used supporting slides but it is indicated in the notes where they did and if you would like a copy of any available slides you can request these by emailing the names of the relevant speakers to GSR.
Plenary 1 – Evidence-based policy making:
rhetoric or reality?
Norman Glass, David Walker, Mark Drakeford and Sue Duncan
Plenary 2
Rt. Hon. Rhodri Morgan AM, First Minister for Wales
Crossing the analytical boundaries
Ruth Kaufman (Operational Research), Sue Duncan (Social Research), Vicky
Pryce (Economics), Karen Dunnell (Statistics)
Plenary 3 – Looking beyond our direct clients
Putting the citizen at the centre
Professor Steve Martin and Ben Page
Evidence:
Production, uses, abuses and neglect in a democratic polity
Professor Colin Talbot
(Norman Glass, David Walker, Mark Drakeford and Sue Duncan)
Norman Glass
Summary of presentation content:
If we check back with the 1999 Modernising Government White Paper we can
see good progress and sound achievements against some of the aims set out
there. Certainly there is more research. It is used better. More piloting
takes place and evaluation happens. More qualitative research is now used,
and there is feedback for those who deliver and implement, but not, perhaps,
enough. There are two challenges to success. One is that although the White
Paper acknowledged that policy making was a continuous learning process,
both ministers and civil servants move around so much that continuity is
lacking. Second is that too often lessons learned are not acted upon or
communicated sufficiently well. Slides available on request.
About the speaker:
Norman Glass is the Chief Executive of the National Centre for Social Research
(NatCen), the UK’s largest independent social research organisation.
Until March 2001 he was Deputy Director in the Treasury’s Public Services
Directorate where he was Chief Micro-Economist. He had a primary role while
at the Treasury in developing the Sure Start programme and was Chair of
Croydon Sure Start until July 2003. He was previously Chief Economist in
the Department of Social Security and the Department of Environment. He
was Chairman of the Economic Policy Committee of the European Union from
1999 to 2001.
David Walker
Summary of presentation content:
The Party conferences seemed to be in praise of professionalism. Even if
at the moment they only see doctors and teachers as professionals, perhaps
in time the bounds will be set wider. Is policy better? Not much sign of
there being an evidence-based policy culture. For example the Alzheimer
drug question – the public are not willing to hear cost evidence in
the context of medicine. And the ‘defining event of the past decade’
(the invasion of Iraq) was made without a strong base of evidence. The centre
of government has not got evidence sorted. Witness CMPS and its iterations,
and the move of GSRU. There is also a lack of analytical focus in the Capability
Reviews. The evidence®policy®implementation process is not adequately
modelled. Although seen as linear, it’s disaggregated at best. GSR’s
role and that of policy colleagues is to build the model. To do this, both
individuals and the profession as a whole needs to get out more. The voice
of reason and knowledge needs to be heard. GSR are ‘keepers of the
flame of reason.’
About the speaker:
David Walker edits ‘Public’, the monthly magazine for senior
public managers and their professional partners, published by the Guardian.
His book with Polly Toynbee, ‘For better or worse: Did Labour deliver?’,
was published last year. David is a trustee of the National Centre for Social
Research and a member of the Franco British Council.
Mark Drakeford
Summary of presentation content:
The context in Wales is different. Compared with Westminster there is a
more even gender split amongst elected members, and with many ministers
in post for longer, continuity is less of an issue. Evidence, however important
in policy making, will always be viewed through the lens of ideology in
a political environment. In the research world exploration of evidence is
often finely grained, but the conclusions are often coarse. This happens
when the quest is for certainty rather than explanation and researchers
trespass too far into decision-making territory rather than providing the
evidence to support that decision-making. Ideology is the hinge around which
politicians use evidence to make decisions. So you need firmer sense of
where evidence belongs and how it plays into the judgements which fall to
those who have to make decisions.
About the speaker:
Mark Drakeford is a professor of social policy and applied social sciences
at the University of Cardiff and the Cabinet’s health and social policy
adviser at the Welsh Assembly Government. He has previously worked as a
probation officer and community development worker. From 2000-2005 he was
joint editor of the British Journal of Social Work and is currently the
European editor of the Journal of Comparative Social Welfare. He has published
extensively in the fields of youth justice, poverty, privatisation and devolved
policy making. His most recent book, ‘Scandal and Social Policy’,
with Ian Butler, has been published by Palgrave and, in its second edition,
by Policy Press.
Sue Duncan
Summary of presentation content:
Don’t treat the Modernising Government White Paper as a cookery book.
We don’t do it with other White Papers or initiatives. It is emblematic
of an approach, it is not a prescriptive methodology. Policy making consists
of ideology, information and inspiration. All three need to be involved
at all stages. GSR needs to be prepared to get involved and ‘get our
hands dirty’ – meaning being prepared to ‘provide what
we can, even if the evidence is incomplete’. GSR should also be involved
in policy making, being part of the creative process so that it becomes
‘evidence inspired policy making’.
About the speaker:
Sue Duncan is the Chief Government Social Researcher and head of the Government
Social Research Service. She has over 25 years' experience in government
social research across four government departments. She is a graduate of
the Universities of Bath and Sussex and an Honorary Fellow of Cardiff University.
She is a Fellow of the Market Research Society, a member of the Social Research
Association and an academician of the Academy of Social Sciences.
Rt. Hon. Rhodri Morgan AM, First Minister for Wales
Summary of presentation content:
Politicians are watched as never before to see that the claims they make
are accurate. There will be increased demand for interactivity in gathering
evidence and those who provide the data also need to understand statistics.
Wales has had a tendency to compare itself unfavourably by comparison with
England. This is a ‘small country phenomenon’ to compare oneself
unfavourably to larger countries – it wouldn’t happen the other
way round. There have been significant improvements in some areas, for example
the rates of child poverty have declined from 37% in 1996 to 17% by 2003,
but was this the ‘low hanging fruit’ that was easy to deliver?
To estimate the future rate of improvement, social researchers need to be
good at futures planning. Looking forward is an entirely different issue
to looking back. For this to work, social researchers must earn the trust
of politicians in order to move forward with their confidence.
About the speaker:
Rhodri Morgan lists health, the environment, European affairs and regional
development among his political interests. Initially a tutor for the Workers
Education Association (1963-1965), Rhodri Morgan was subsequently a research
officer, an economic adviser, an industrial development officer and head
of the European Commission Office in Wales. He was elected to Parliament
in 1987 but decided not to stand in the 2001 General Election. He was elected
Assembly Member for Cardiff West in 1999, becoming First Secretary of the
National Assembly in February 2000, with the title of his office changing
to First Minister in October of that year.
Ruth Kaufman (Operational Research), Sue Duncan (Social Research),
Vicky Pryce (Economics), Karen Dunnell (Statistics)
Ruth Kaufman
Summary of presentation content:
Operational research (OR) is about applying scientific methods to solving
management problems by applying scientific processes to improving improve
organisational performance. It is defined by purpose and area of activity,
not by distinct methods or domain knowledge. OR analysts may use a range
of tools varying from problem structuring, through decision analysis and
forecasting, to mathematical modelling. (Examples were given of the kinds
of modelling and analysis that OR people do, for those in the audience who
had never come across OR before). Where policy-makers are trying to change
the world, and where specialists are researching/ understanding the world
to provide an evidence base, OR people are helping to make change work.
Good evidence from other disciplines improves OR models.
Conversely, good OR models can help other analysts' evidence be better used by policymakers. Operational Research is a tool to 'help make change work'. Cross disciplinary working is worth doing whenever appropriate, not just to share information across disciplines but to help identify what question should be asked – it helps deal with problem of not knowing what we don't know. It helps identify what question should be asked. Shouldn't just be across the disciplines represented on the panel: there are many other specialists where cross-disciplinary work is equally important, for example accountants, IT. OR grew out of the need to bring different disciplines together to solve novel problems during World War II. There is a cycle of development as disciplines come together and create new disciplines. Social researchers should therefore define themselves on the basis of solving a type of problem, working across boundaries as necessary to do this. They should aim for flexible teams and porous boundaries. Slides available on request.
About the speaker:
Ruth has spent most of her career working as an OR analyst in the public
sector with London Transport, London Electricity, the Department of Health
and the Export Credits Guarantee Department (ECGD). She has headed the OR
team there since 1996. In the last two years she has led ECGD’s combined
Strategy, Business Change and OR activities, and is now a member of ECGD’s
executive committee.
Sue Duncan
Summary of presentation content:
All social scientists should also be considering how to work better with
natural scientists – but first, let’s get the social science
house in order. Most social policy is about behavioural change so disciplines
need to come together – but how? The usual approach is organisational
change, but this is not the only answer. Each discipline must have a working
knowledge of the others and must know what resources are available. So whoever
is the first point of contact in the analytical community should be able
to answer the request or quickly identify who can. It’s as much about
culture as organisation. Those who use our outputs need to understand what
we offer. Professional Skills for Government means generators and users
of evidence have a job of education to do for each other. Is this, though,
the demise of distinct professions? No. It is even more important in cross-disciplinary
working to know the particular skills need for each job. The analogy can
be drawn with the medical profession, where the patient ultimately still
wants to see a specialist appropriate to their condition. So the trick will
be to keep the category together whilst still maintaining the specialisms
within it. The danger of a generic ‘analysts’ badge is that
it would lack meaning for the wider academic and research community, which
in turn would weaken the professional reputation of the disciplines in the
eyes of their clients and make it harder to recruit the best into government.
About the speaker:
Sue Duncan is the Chief Government Social Researcher and head of the Government
Social Research Service. She has over 25 years' experience in government
social research across four government departments. She is a graduate of
the Universities of Bath and Sussex and an Honorary Fellow of Cardiff University.
She is a Fellow of the Market Research Society, a member of the Social Research
Association and an academician of the Academy of Social Sciences.
Vicky Pryce
Summary of presentation content:
Shrinking departments mean a greater need for specialists in professional
groups, as there is likely to be reduced capacity within individual teams.
The real breakthrough will be when members of the Government Economic Service
and others can be seen not solely as analysts, but as people who can, if
they want to, move across into policy - and in so doing help the policy
audience be a much more intelligent customer for analysts. It is very important
to retain a distinct professional identity both within government and within
the profession, but must be seen as part of a wider group. The issue, however,
is that another discipline may have a voice or a 'seat at the table' above
you in your organisation. So, for example, economists may have to report
in to the chief scientist rather than a more senior economist.
About the speaker:
Vicky Pryce is the Department of Trade and Industry’s Chief Economic
Adviser and Director General, Economics and Deputy Head of the UK’s
Government Economic Service. She is on the Council of the Royal Economic
Society and has recently been elected a Fellow of the Society of Business
Economists. Vicky is also Visiting Professor at Cass Business School (formerly
City University Business School). She has also been elected on the council
of the University of Kent, and on the board of trustees at the RSA.
Karen Dunnell
Summary of presentation content:
There are lots of examples of analysts working together in multi-disciplinary
teams to good effect. An example of where this happens within the Office
for National Statistics (ONS) is work on pensions. Issues which need to
be considered here include saving, healthy life expectancy, inequalities
in pension provision etc., requiring the skills of economists, statisticians
and social researchers. Applying this more generally across government requires
commitment and leadership. There is a tendency for professional groups to
work in a silo-ised way, although this is definitely changing, as is evident
in the work on Social Exclusion, SureStart and Neighbourhood Statistics.
Having one Government Analytical Service, in terms of an organisational
structure, may be an optimal way forward. As people progress in their careers
they will inevitably find that their roles cross boundaries with other professional
disciplines. Essentially, all that is required is for people to identify
themselves with a distinct professional group. This model would help with
the secondment of staff into different areas,
engendering a culture of knowledge sharing as we progress with the Professional
Skills for Government agenda.
About the speaker:
Karen was appointed National Statistician and Registrar General for England
and Wales in September 2005. She started her career as a health care researcher
and lecturer, joining the civil service in 1974 to work on social surveys.
In 1999 she moved to the centre of the Office for National Statistics (ONS),
leading the division that supported the ONS executive. She prepared and
launched National Statistics in 2000 and paved the way for Len Cook, the
first National Statistician. She was promoted to look after a wide range
of social statistics and in 2002 was appointed to the ONS executive.
Professor Steve Martin
Summary of presentation content:
In putting the citizen at the centre, professor Martin raised three questions:
Why? How? And what are the implications for social research? He described
the three models. Firstly, give choice by putting competition into the system
as a driver of choice. Secondly co-production by neighbourhoods and groups.
This supports local democracy and civic renewal. Finally the citizen-centred
model which sees collaboration between providers rather than competition.
In offering choice it is not realistic to have several suppliers in many cases, so there is no choice. So the need is to have one public service. But this requires consumers to have a much stronger voice in the absence of real choice. ‘Double devolution’ from central to local government and from there to citizens can put councillors in the role of community champion. Offers potential way to re-engage citizens with the institutions of government and deliver sustainable service improvement. However, there could be a loss of economies of scale; do people want to take decisions locally? Can it work at a neighbourhood level for things like prisons, schools, hospitals? The collaborative model has not really happened because it is costly, time-consuming and the culture is against it – incentives tend to be about hitting your own targets.
Does government research need to follow the double devolution of power? Is research as a departmental, expert, central service sustainable? A devolved, or doubly devolved, service would make it difficult to get an overview and too much small scale research would inevitably result in wheel re-invention.
Can we expect the public to be informed enough to participate in research? Do we need citizens to be commissioners of research? We know that the centre is asking for customer satisfaction data. So we need smarter data – not just what people think, but why. Capture a range of voices and have greater consistency of measures over time, whilst also offering more rapid and regular reporting. This can be achieved by better management of existing data, not just commissioning new surveys. Finally, it is not just the citizen/consumer whose voice should be heard but also that of the staff delivering services, who need to be engaged. Slides available on request.
About the speaker:
Professor Martin is Professor of Public Policy and Management at Cardiff
University and the Director of the Centre for Local and Regional Government
Research where he is leading a multi-million pound programme of research
on local governance and local service delivery. The programme is funded
by the ESRC, Department for Communities and Local Government, Welsh Assembly
Government and the Audit Commission. Professor Martin has conductd a wide
range of studies for UK government departments, the EU and other agencies.
He is a director of the Improvement and Development Agency, a member of
the Public Sector Management Wales Advisory Board, associate editor of Local
Government Studies and a member of the editorial board of Public Money and
Management.
Ben Page
Summary of presentation content:
MORI plc does more work for the UK public sector than its French parent
company does in the whole of the rest of the world. The perception gap means
expectations of services are worse than experiences of services. Expectations
of services are only optimistic overall for education, and even that is
declining. Governments are generally not well liked, so this political perception
rubs off on the services it provides. Hence perceptions of local services
are higher than of national issues. So we need to un-pack research data
locally. Media and the staff who work in providing services also have an
effect on perceptions, so these dimensions must be measured. Government
needs to illustrate success with positive stories. But is has failed to
take staff with it so, despite increased investment in public services,
not enough of the staff who work in them are advocates for their service.
Slides available on request.
About the speaker:
Ben Page is chairman of the 130 strong Ipsos MORI Social Research Institute
and managing director of Ipsos MORI Public Affairs. He joined MORI in 1987
after graduating from Oxford University. A frequent writer and speaker,
he has directed hundreds of surveys for UK local and national government
and large service providers examining quality of life, service delivery,
customer care, communications and democratic deficit. Since 1992 he has
worked closely with ministers and senior policy makers across government,
leading on work for Downing Street, the Cabinet Office, the Home Office,
ODPM and the Department of Health, as well as a wide range of local authorities
and NHS trusts.
Professor Colin Talbot (Manchester Business School)
Summary of presentation content:
Taking a broad approach to evidence sources in the context of public service
delivery and the differences between evaluation/research and performance
management, what are the implications that these approaches may have for
the evidence base and the accountability of government?
Two sorts of evidence which rarely come together are evaluative research of public policy, and performance of public services. There is a huge amount of creators and transmitters of research. Lots can go into a black hole (for example, the 15 reports into the prison service, mostly not acted on). Evaluative research is episodic, it is about policy/programmes and it is qualitative. Performance research has an organisational focus, it is periodic and quantitative. Characteristics of the two types are illustrated in this table:
| Evidence | |
| Evaluation and research | Performance |
| Policy programme focus | Public sector organisations focused |
| Episodic (i.e. tends to come to an end) | Periodic (i.e. covers fixed periods) |
| Tends to be qualitative | Tends to be quantitative |
| Piecemeal | Universal |
| Researchers (producers) | Managers (producers) |
| Policy makers (users) | Managers (users) |
Parallel or converging systems/approaches?
Evidence suggests that the two approaches to evidence are running in parallel
which is resulting in two separate systems of evidence gathering and analysis.
Although a great deal is known about the demands for evidence, much less
is known about the use of evidence and the impact that evidence has on policy-making.
If evidence is influencing decision-making, it was suggested that there
is a need to ensure that data quality is audited.
There is a recognition that the development of different performance systems
across government is not effective and that, if public services are to be
managed effectively, there is a need for a more integrated approach to performance
management across government.
Data we produce will be used selectively by lobby groups, the media etc. But don’t use this as a reason not to produce it. Issues are quality of data, selective manipulation and, perhaps worst, neglect of data.
Discussion points:
Experiences of frontline service managers illustrate the competing demands
that are placed on public services to collect performance management information
and the costs (financial and human) attached to this. Demands for information
sometimes conflict with the direction in which public services are being
steered by government.
It was argued that there was no philosophical differences underpinning
possible philosophical differences between evaluation/research and performance
management.
Qualitative evidence versus quantitative? Ongoing debates centre on the
largely quantitative nature of performance information and the qualitative
nature of evaluation material. Although there is a degree of convergence
in the use of quantitative and qualitative information in practice (e.g.
the British Council has sought to develop a performance management system
that combines qualitative and quantitative information where statistical
data is supported by narratives), it was suggested that ongoing debates
continue to focus on the divergence of these two types of information. Slides
available on request.
About the speaker:
Professor Talbot holds the chair of public policy and management (PPM) at
Manchester Business School, where he also co-directs the Centre for PPM.
He is also director of the Herbert A Simon Institute for Public Policy and
Management at the University of Manchester, which brings together four schools,
half a dozen centres and institutes and around 200 scholars across the university.
Before becoming an academic he worked in a variety of private and mainly
public sector posts. He has particular interests in civil services, agencies,
performance and international comparisons. He advises the National Audit
Office and has given evidence to the Treasury Select Committee as well as
the the Public Administration and the Wales Select Committees.