Prioritising problems for public engagement

Wellcome’s Public Engagement team has been exploring new ways of funding public engagement. One approach they’ve considered is going on collective missions, which involves helping many actors work together to solve a particular problem. The team knew they wanted to build a funding programme around this approach, but first, they needed to select the right science and health problem for it.

Drawing on their previous experience funding and leading on public engagement across a wide variety of science and health domains, the Public Engagement team shortlisted four problems with potential to be addressed through a collective missions: vaccine hesitancy, epidemics, the gender data gap, and shifting to low- and no-meat diets.

Our role was to help the team deepen their understanding of each of these problems, compare them, and select one or two to design a public engagement programme around.

We started comparing the problems from the very beginning of this project. At first, we used prioritisation as a way of surfacing assumptions and gaps to highlight where more research was needed. As our understanding grew, new research insights increasingly fed into more evidenced and better reasoned judgements on each problem’s suitability for the task at hand.

Building a basis for comparison

To rapidly learn as much as we could about each problem under consideration, we read publications, interviewed experts, and sought out opinions, counterpoints, and indications of whether and how public engagement could contribute to solutions. We kept track of our evolving understanding in problem briefs – short documents that summarised a problem, existing work addressing it, pros and cons of the Public Engagement team getting involved, and opportunities for the team to have an impact. These problem briefs were useful for refining our understanding and sharing what we’d learned with the Wellcome team. We also used them as a stimulus for interviews with experts, which was helpful for getting concrete feedback on our ideas.

Specifying criteria

Our earliest prioritisation work focused on developing criteria with which to evaluate the problems. Having a clear set of criteria can support more open and constructive conversations and facilitate decision-making because it means everyone has a shared understanding of what is important.

We have developed a set of criteria that we’ve used across many projects. Through conversations with the Wellcome team, we customised these criteria and ended up with a list of eight grouped into three categories: those that indicated a good problem, a good problem for public engagement, and a good problem for Wellcome.

Scoring against criteria

In the first workshop, we guided members of the Wellcome Public Engagement team through a criteria scoring process. Each team member independently scored each problem against each criterion, and then we added up the scores to rank the problems.

This scoring wasn’t meant to lead to a final decision, but it ensured that each criterion was being properly considered. The scoring also highlighted disagreements within the team which could then be discussed.

Going through this process led to one of the problems being dropped, and we got a clearer understanding of where more research was needed.

Making head-to-head comparisons

During the research, we had found it useful to stage head-to-head debates with Richard picking one problem and Andrea arguing in favour of another. Recognising that this format was great for uncovering key issues underpinning the ranking of each problem, we incorporated it into the problem briefs in the form of a 'pros and cons' section. We also took this approach in the final prioritisation workshop with the Wellcome team.

At this later stage, narrowing the focus to decision-critical issues was more important than considering each criterion in turn. In the workshop, we asked the team to make a series of holistic judgments on all possible pairs of problems, individually determining the winner of each direct comparison. This method resulted in a list of problems prioritised according to how well each performed against each of the others.

Going through this process ensured that each participating member of the Wellcome Public Engagement team was equipped to make an informed, confident decision on which problem to select for the collective missions programme. After a final discussion, they chose to focus on one problem: vaccine hesitancy. During a period of further research, this was then reframed as vaccine uptake. We're continuing to work with the team, and are now helping them design a programme around this problem.

Although the remaining three problems weren't selected for this particular programme, the research and prioritisation that has been done provides a starting point for scoping additional programmes. For instance, the Wellcome Public Engagement team can use this background to build on existing collaborations or respond to emerging opportunities within and beyond the organisation.

It's been exciting to help develop a funding programme from the early stages of decision making, where we think prioritisation plays a crucial role in increasing potential for positive impact. This project has also been exciting because it's given us an opportunity to refine our methods in several different ways. Three important things we've learned are:

Debate is a valuable method for prioritising problems

After doing enough research to get a basic understanding of a problem, engaging in debate can help to surface key considerations for prioritisation. By poking holes in each others’ arguments, you can uncover more issues faster than one person thinking by themselves.

Problem definitions are bound to change through research

For each of the problems, our research (and especially our interviews with experts) challenged us to consider whether the framing we started out with was right. For example, we found ourselves asking:

• Is vaccine hesitancy the problem, or is the problem more broadly about vaccine uptake?
• Is the gender data gap more accurately a sex and gender data gap? How does this relate to biomedical research versus public health research?
• Would we want to encourage the reduction of meat consumption if that caused people to switch from red meat to chicken and fish, which may have a worse impact on animal welfare and certain aspects of the environment?

It is possible to compare very different problems

It’s difficult to compare problems as different as the ones we were considering for this project, but taking a systematic approach is still better than making a quick, intuition-based decision. Through our research, we were able to uncover key ideas and arguments that were essential for making a good prioritisation decision.

Are you wondering how to prioritise problems and define a focus for your funding programme? Tell us more about it – we’re here to help!