MYTH #6: The 1-6 score on the review determines how a proposal does at panel

I am halfway through sharing the most common funding myths, and I have found that number six is a belief that many people don’t like to let go of. However, there are a few things that tip this statement into the myth zone.

First, 1-6 is not a score, at least not in a traditional sense. A score implies that the number is used to determine something. A test score may indicate if someone passes or fails. A score in sports indicates the number of goals, baskets, or runs and therefore which team wins or loses. At EPSRC, the 1-6 evaluation is a summary mark: it summarises how supportive (or not) a reviewer feels towards the proposal.

To be sent to an EPSRC prioritisation panel, a proposal must have at least two supportive reviews. This is based on the content of the reviewers’ comments and not purely the summary marks each proposal receives. This is because the marks are subjective: some reviewers will never give a 6 and others will never give a 3. But, as a rule of thumb, 1-3 marks are considered unsupportive and 5-6 are considered supportive. Summary marks of 4 can often go either way, and the portfolio manager processing the proposal will need to decide if the reviewer is supportive or not.

On top of all this, the comments should be in alignment with the marks. A 6 with highly negative comments or a 1 with a supportive evaluation is likely to be questioned by the portfolio manager.

Second, it’s the reviewers’ comments—and how well evidenced they are—that the panel uses to decide their ranking. For example, a review that has given a 6 but which doesn’t explain how the proposal meets the assessment criteria or why the project should be funded could be ignored when the panel makes their decision. The inverse of this is also true: a reviewer who has given a low mark could be completely dismissed by the panel if the comments have not actually bothered to highlight why the reviewer is unsupportive. In other words, all reviews should provide evidence for why the reviewer has evaluated the project as they have.

So, while these numbers are not meaningless, they are not as meaningful as many researchers seem to think. Focus on the content of the review and writing a strong PI response rather than counting your 6s before they go to panel.