Government Social Research
Time is often the key factor that dictates the resources needed in an REA but the quicker it needs to be completed the more people and money it is likely will be needed. This section discusses these three resource elements and also provides guidance on the information management requirement for REAs
How many people are needed?
The number of people needed in the REA team and their roles will depend on the type of REA that you want to carry out and how quickly you want to complete it. Some teams will expect members to be involved in all stages of the REA whereas others will have people who specialise in different aspects (e.g. searching).
Who should be in the REA team?
The section on forming the team covers the types of skills people need to carry out an REA, including training for staff. However, these skills may not be available within your department or agency and there may not be time to carry out training prior to the REA starting. In this case parts of the REA can be carried out externally.
This often involves the main REA team specifying the scope of the REA and writing the protocol.
Stages of the REAs that might benefit from specialist expertise are:
Brining in external expertise should be planned as early as possible in the project. Since the time between commissioning and commencing REAs is often short – days or weeks, rather than the months that are more common with large research projects – it can be difficult to recruit researchers quickly to work on an REA. For this reason, organisations which undertake this work, will either need to have staff in place who can be pulled into working on an REA at short notice, or a pool of more casual staff who are available to work as and when they are needed.
It is not possible to say how long each stage of an REA will take, since there are so many possible variables. However, if an REA is given four months from its start to the completion of a draft report, the following timeline might be useful to consider:
Task |
Time |
|---|---|
| Forming the team, consulting users, writing protocol | Month 1 |
| Searching screening (and writing map in two-stage REAs) | Month 2 |
| Data extraction and quality assessment | Month 3 |
| Synthesis and writing the draft report | Month 4 |
Within this basic framework, the resources available will dictate the amount of work it is possible to complete in each stage. For example, with very clear definitions of inclusion/exclusion criteria and an experienced team, it is possible for one researcher to screen up to 1000 titles and abstracts per day. However, with less experienced researchers, or more complex/less well defined concepts, this can fall to less than 300. Therefore, the search strategy needs to take account not only of the number of references that can be managed, but also with regard to the ease with which the team is able to agree on whether or not a particular study meets the inclusion/exclusion criteria of the REA.
One other important area that can lead to delay is waiting for libraries to obtain the full studies that you have been screened for inclusion in the REA. Some papers can take a considerable time to arrive and so it is important to ensure you have a librarian/information specialist in the project as early as possible to manage the retrieval of information effectively (see screening and selecting studies)
The costs of an REA will depend on the type of REA that you are undertaking and the length of time that it needs to be completed in. Typical elements for a budget for an REA will include:
Contracting out an REA
The cost of contracting out an REA will depend on the volume of literature, how dispersed it is, how easy it is to locate, how easily defined the intervention is, how far back the search needs to go, how quickly the REA needs to be done, how much input from experienced researchers is needed and how experienced the reviewer is. As an example, the average cost of an REA is around £40K. However the range is large: from £20K up to £70K.
Good information systems are essential for an REA to be completed successfully. There are two ways of organising information for use in an REA: using either a sequence of generic software packages, or specialist reviewing software.
Standard office software
Standard office software can be used to manage data generated from an REA, for example, using Excel to record searches. This option will cost less than specialist software since it is pre-installed on most PCs. However, the purchase of more specialised software can save time by making the REA more efficient.
Specialist software
There are specialist software packages for systematic reviews that are also suitable for REAs. Some are designed to manage the data through the entire project, whereas others perform specific tasks (such as synthesis). ‘Chaining’ a variety of software applications that perform specific tasks is a common means of managing REA data. It is effective, since it uses the strengths of each application, though it does require good documentation. On the right is a list of software applications and the tasks that they are able to perform.