What did EPSRC do? Part 4: Impact, Importance and the Shape of Capability
I think the most contentious things that the EPSRC has done in it recent structural changes relate around the RCUK Impact agenda, common across all research councils, and its implementation of a a ‘shaping capability’ program, looking at focussing research funding to research areas of particular need or importance.
In my opinion these changes aren’t such a huge deal but the way in which they have been communicated has been poor, and I’m not really completely sold on the specifics on the implementation.
Pathways to Impact
Pathways to Impact is a RCUK mandated change to grant proposals, and as it has been communicated to me, the idea is that in order to justify what research scientists get money for then we have to give some indication as to what use that research is to be put to. The big initial issue with this was the assumption that ‘impact’ meant economic impact – and this has resonances with problems with the introduction of a measure of impact within the UK’s Research Excellence Framework (REF) assessment process that all Universities are currently preparing for. However, I think it is clear now that, while I’m sure RCUK would be happy if every grant was going to spawn a spin-out company, impact has a much broader scope. It includes ways in which your research will underpin other research areas and developments, how it might be used by other researchers and how it can impact on the wider community, for example.
The practical consequence of this is that it forces you to think about the utility of what you are asking the taxpayer for money to do, and to think about the non-economic benefits as well as economic benefits of the proposed research. I don’t really think that anyone can have any significant objection to this process. There are issues however. Firstly, most people did this anyway. You’d be daft to write a proposal which did not have some form of justification as to why you needed the funding. This would be taken into account by reviewers, and the strength of your science would then be the major factor in reviewers comments. What is now asked for, is a two page discussion of your proposed pathways to impact. For a NERC ‘Small Grant’ which is only two pages long, and even for a standard six page EPSRC grant, this seems somewhat excessive.
More generally, with the idea of justifying what your impact might be, the question is: who am I justifying it to? The idea that has come from EPSRC, and I think I’ve heard David Delpy discuss it in these terms, is that in order to maintain science budget spending, work has to be justified to Whitehall and the Government. But I very much doubt that anyone other than the referees and maybe someone at EPSRC will ever read the statements. They are not published. So it just feels like a lighter touch could have been employed; that the extra burden on the grant writer could have been minimised, and impact could have been folded into the structure of a grant in a more seemless way. At the end of the day what it means is more work for grant writers and reviewers, for which I see no real need. The usefulness, or lack thereof, of such statements is also touched on by Athene Donald in a blogpost from last year: “Very few referees comment specifically on the mandatory (for UK research councils) ‘pathways to impact’ statement”. Maybe this has changed, but that is how I tend to deal with them, when I review, so it does beg the question, why do we go to so much effort?
The other complaint about the Pathways to Impact statements was that they are an invitation for hyperbole and exaggeration about the significance of your work. I think this is a legitimate concern – I have seen many grant applications prior to the Impact statements that were very good at promoting their work in unrealistic ways, and that by flagging up some big numbers from international reports and market surveys made their end goal look far more impressive than it really was. The issue here, I feel, is that weight is given to the statements when reviewing, and while most reviewers are canny enough to spot this type of stuff, it can lodge subconsciously in the brain, and maybe you rate a ‘full-on’ impact statement grant a little bit better than a more sober and realistic one.
When writing an EPSRC grant application, one has to write a short ‘impact statement’ on the submission form (on JeS), and accompany this with a full up to two page ‘Pathways to Impact’ statement as part of the grant. My solution would be to stick with the short form version, have some consideration of it in reviewing to make sure the applicants are taking it seriously and then publish it if the grant is awarded for all to see. This would satisfy all the points I think that RCUK want to make, and would reduce the overall burden on applicants and reviewers.
I have yet to write a grant making use of the newly required ‘national importance’ statement. I have been to an EPSRC workshop on the issue, and am not entirely sure I am much clearer about what they really mean. This in part is perhaps because EPSRC don’t seem to really know what they mean either. This statement within a grant is a short form, perhaps a few paragraphs (so light touch), and “reviewers will be asked to consider national importance as a major secondary criterion, after research quality, in their assessment.” So it would seem to be important in terms of your funding success. So what is it? The controversy comes form the fact that it is a prediction looking forward to the next 10-50 years…which seems a recipe for hyperbole and spin. From the EPSRC webpage on National Importance:
National Importance is the extent, over the long term, for example 10-50 years, to which the research proposed:
- contributes to, or helps maintain the health of other research disciplines contributes to addressing key UK societal challenges, contributes to current or future UK economic success and/or enables future development of key emerging industry(s)
- meets national strategic needs by establishing or maintaining a unique world leading research activity (including areas of niche capability)
- fits with and complements other UK research already funded in the area or related areas, including the relationship to the EPSRC portfolio and our stated strategy set out in “Our Portfolio”
The problem here is that the language is unclear. How can one predict over a 10-50 year timescale how your work “contributes to current or future UK economic success” ?- that doesn’t really make sense to me, or how can it “fit with and complement other UK research already funded in the area”? The language seems confused.
More confusing are the suggestions made by various senior researchers giving their views on what should be in such a statement, and how they would review them. These all seem to take slightly different viewpoints, and it is clear there is no real coherent idea of what such a statement should be. I mean how can there be? What will be of national interest/importance in 10 years time is very difficult to guage. If I went back thirty years and I was developing new mathematical models for financial derivative analysis, I could probably make a good case for making someone lots of money ten or twenty years hence. Pushing it to thirty, and I’m not sure that provoking a world wide financial meltdown would have been something I’d strongly want to advocate. To me, and to many others, this is just a silly inclusion.
But the thing that annoys me here, is again the question of ‘why?’ I have no truck with making suggestions about the impact of my work, or how I can seeing it playing out in the future. But why give these things such weight, and why the need to drill down and specify them in some a niche way without seeming to have a clear idea about what they mean. Who is this stuff for?
I note that the EPSRC Council have a letter in the Telegraph today, in which they say, “With a spending review possible in 2013, this is a time for the research community to work together to explain why science and engineering is so important for the UK”. I think that the Pathways to Impact and National Importance statements are at the heart of this. The problem is, that unless David Delpy is literally going to copy and paste these components into a report for Ministers, then this seems a poor way of going about telling a coherent and upbeat story.
EPSRC have undertaken a shaping capability exercise recently, which was designed to try and align the funding portfolio more close with areas of ‘national importance’ (whatever taty may be) and UK strength. This could have been a fairly radical change, and EPSRC has come under a lot of flack for how it has gone about its ‘grow, reduce, maintain’ program of reviewing different portfolio areas. The outcomes for physical sciences can be found here. [As a side note, finding stuff on the EPSRC website can be a real pain]. Most things stay the same, a few grow and reduce. Clearly those in an area of reduction may feel angry/worried. EPSRC’s failing here has come in how it has communicated how it has come to these decisions. The suspicion has been that in many cases decisions has been taken by non-expert EPSRC staff, and that consolations have not been taken as widely as they should. Some of this criticism seems justified, although some smacks of, ‘well they didn’t ask me!’
Once again I would ask whether these changes really serve anyone well. What appears to have happened is that small group of areas will have funding cut, but I question what the real practical consequences of this will be, as the area definitions are often quite woolly, and most grants span more than one. When we hear of a grant being returned due to its subject area being in a reduced topic, or when a panel is directed to fund a grant in a ‘maintain’ or ‘grow’ topic over one in the ‘reduce’ pile then we should get angry. EPSRC could have cut many more things, and really shaped the portfolio, but it seems to chickened out. My opinion here is that we should fund good stuff equally, but if a decision was made to shape, then, had I been in charge, I think I would have been more aggressive. So maybe it’s just as well I’m no where near the reigns of power.
So to end this little summary of what EPSRC has been up to (I will omit detailed discussion of some of the other minor points I raised in the first article), my personal belief is that EPSRC is doing a tough job, and is having to make tough decisions. I think the current disgruntlement with this process will pass in due course, but it would be nice to have clearer information about why the specific details around some of the implementations have been the way they have. I see no issue with discussions of impact, am skeptical about national importance, will lament the loss of grant funded PhD students and think the limited submission policy could be improved. Aside from that I think that EPSRCs community needs to try and keep communicating the issues we have in a sensible way, and that EPSRC needs to learn how to communicate more effectively.