This weekend I received the email below regarding the current policy to allow ONLY A SINGLE REVISION for a given grant at NIH. If you feel strongly that this policy is misguided (particularly in the current funding environment)- please respond to Dr. Robert Benezra by Email at: r-benezra@ski.mskcc.org
**UPDATE: I just received an email with a draft petition with nearly 1000 signatures attached. The text of this petition is posted here. If you would like to be added to this petition please email Dr. Benezra directly.
Dear Colleagues,
I am writing to solicit your help in changing a new NIH policy that I believe will have an enormous negative impact on our field. As most of you know, a recently adopted rule states that if a grant proposal is not funded on the first submission, only one revision can be submitted with the same specific aims. If that revision is not funded, the proposal must be “substantially” changed. As far as I understand, the rule was adopted to discourage “serial resubmitters”. While such a policy could make sense in an era of reasonable paylines, with the projected budgets rumored to be funding at the 7th percentile in some sections, this could have a dramatic and I would argue devastating effect on the research efforts in this country. Consider the following:
The rule will have a disproportionately negative impact on young investigators with early stage and therefore less diverse programs, or more senior investigators who also have more narrowly focused programs. How can a young investigator, for example, who is just starting “substantially” change their aims when they have to focus their efforts on a very limited number of projects undertaken with limited funds and staff. These people are often hired by senior faculty on the basis of their first projects and to be told they must change on the basis of applications that might fail despite being ranked better than 90% of grants submitted, seems patently absurd. And worse, it is likely to be profoundly discouraging and destructive.
All of us who have sat on study section know that we cannot distinguish a 15th percentile grant from a 5th percentile grant. It is simply beyond the resolution of the process. Therefore, this new rule will have the consequence of redirecting the science of many of our very best scientists on the basis of what will essentially be an arbitrary criterion.
The meaning of “substantially changed” has not been clearly defined. Program Officers themselves are not sure what this term means and are not being given adequate guidance. I have heard things from “51% different”, change the tissue or cell type you are working on, any aim included in either the first application or revision cannot be included, etc. We need clear and unequivocal guidance on this point, and I would argue we need it immediately as “new” applications are being prepared by a large number of investigators at this time.
The alternative that I advocate would be to go back to a system where at least 2 revisions of the same application would be allowed. While we will still obviously lose some superb applications if the pay line stays where it is, I think this would provide a much fairer assessment of the research proposals received by the NIH.
My intention is to let the feelings of a large number of scientists on this subject be known. If you are willing to sign an email that will be sent to both Francis Collins and Tony Scarpa (Director of Center for Scientific Review) that raises these points, please let me know by simply responding to this email and (if possible) forwarding it to 10 people who you know (not on the current recipient list) that might also want to sign. If I can accumulate a large enough number of signatures (100-500, say) I will draft a letter and send it first to all who have expressed interest in signing to get feedback.
I must say, I am not generally prone to such activism but I think things have just gotten to the tipping point.
I look forward to your responses.
(I’ve removed the institutions of the original signers of this email, because these are their personal views)
Blue Lab Coats 
There is a lot of fundamental misunderstanding going on in this letter. We can start with this:
No, the rule was adopted to try to stop the common study section behavior of putting meritorious applications in a “holding pattern” for one or two resubmissions, so they can fund the applications that have already been in a holding pattern. The other change made by many ICs to prevent the “holding pattern” is the adoption of different paylines for A0 and A1 applications, with the former substantially more generous.
More importantly, however, there is a serious delusion that underlies this letter. There is only so much money available to fund competing applications, and the only effect changes in peer review in terms of actual funding of such applications could possible have is a change in which applications get funded. So all this talk of “meritorious applications going unfunded” is nonsense. Limiting resubmissions can’t possibly change the number of “meritorious” applications that go unfunded.
The letter authors seem to have forgotten that–while they may feel put upon that they only get a single resubmission–all their competition also only get a single resubmission. The playing field is still even, but in a context that should make peer review more efficient by substantially reducing “holding pattern” study section behavior. It will also reduce the PI behavior in response to “holding pattern” of submitting half-baked proposal they *know* aren’t fundable in order to “get in line”.
Finally, as far as New Investigators/Early Stage Investigators, that is a red herring. Their grant applications are reviewed separately in study section, and are supposed to be assigned criterion and impact scores by comparison with each other, not with established investigator grants. And even more saliently, all ICs have substantially more generous ESI paylines than for established PIs, in some cases more than *twice* the established PI percentile cutoff.
“There is a lot of fundamental misunderstanding going on in this letter. We can start with this:
As far as I understand, the rule was adopted to discourage “serial resubmitters”
No, the rule was adopted to try to stop the common study section behavior of putting meritorious applications in a “holding pattern” for one or two resubmissions, so they can fund the applications that have already been in a holding pattern.”
Ok, Whatever the reason- the end result ends up being that a 8.1th percentile proposal drops off the cliff after 1 resubmission. To me that is letting a meritorious application go unfunded.
However- I do agree with you that just not having enough $$ is the fundamental problem. Got any ideas on how to fix that one?
Any researcher who cannot come up with three more Aims that allow her to circle back around on the topic she is really interested in pursuing probably doesn’t have enough imagination to justify funding anyway. 😛
DM- What constitutes ‘new’ is different in each person’s definition. And secondly- if you want to have a different proposal in each of the three grant review rounds for NIH per year- you are ALREADY spinning your preliminary data in multiple ways. Usually more than 3 ways if you are also writing R21s simultaneously, and submitting to multiple agencies… (each with a different focus)..
right, so if you are taking three different proposals to each of the three rounds…..
I’m just not seeing where the end of the A2 is the end of the world here. What difference does it make? The “third chance” that you imagine is predicated on the prior times when budgets were better. Look, reviewers know what it means when they assign a not-fundable score to an A1, just like they knew with an A2.
Pingback: Everybody deserves a second chance? | Tales of the Genomic Repairman
The end of the A2 is not the end of the world, but it is just one more absurdity that now is forcing everyone to try to think about what kind of science reviewers *might* like to see, instead of just doing science. Everybody now spends time working on feasible preliminary data instead of following up that supercool observation that doesn’t quite fit in the box but could tell us something unexpected and interesting. And after one revision, it’s time to come up with another set of preliminary data that might go in a different direction than the science practically dictates that it should. And I agree that reviewers know what it means when they assign a not-fundable score to an A1, and sometimes they are right. But sometimes they’re not, and that’s another 6 months and a whole lot of time and effort down the drain. And if you think that it’s just whiny losers who can’t get their grants funded signing the petition, you’ll think again when the petition is made public and you see faculty from every major research institution, Howard Hughes investigators, people with multiple R01s, etc.
Who said anything about people who can’t get their grants funded? Or whiny losers for that matter?
I would like to know CPP’s and DM’s current funding status. Not that this would bias their thinking on the issues, but it seems relevant. I know from my readings that DM is an advocate for breaking up the old boys network. But I have to wonder if DM is funded under the current regime/protocol/whatever, is his/her defense of the new system in part a protecting what’s mine response. ‘This worked for me, and Im totally awesome and unbiased and concerned about new scientists, therefore the system is fine” mentality. This is a sticky situation because anyone previously funded but now not funded and anyone continually funded have biased views about the system. Since we are talking about peoples’ careers and the careers of those within their labs, this is not a trivial bias either way.
[DrugMonkey February 14, 2011 9:35 pm
Who said anything about people who can’t get their grants funded? Or whiny losers for that matter?]
It’s not personal, DM, but rather “you” in aggregate. 🙂 If only I had time to root out every comment that refers to whining and losers in all the threads that touch on NIH peer review and funding to make my point…. Unfortunately, I have work to do and deadlines to meet. You are a smart guy though; I’m pretty sure you read all those threads too since many of them were in response to your posts.
it is mostly PP that is on about the whiny losers, bugdoc. I am definitely on about the “whiny” but I tend to reserve that for those that I think had their chance (generationally or whatnot), are in a good enough situation even if they don’t get more grants (i.e., hard money for teaching, made tenure) and have no perspective about what the kids these days are facing.
Lorax, perhaps one day we can meet up and you can review my funding history in excruciating detail. As I said somewhere or other, I think it best to line up all one’s favored solutions to “the problem” and assess how many tend to help and how many tend to hurt one’s own research program. If your latter list comes up empty, well……. My list does not come up empty, I can assure you.
I am not an advocate of “breaking up” anything. I am an advocate of more fairness in review within a couple of key areas, mostly for the young PIs. Also for clarity and consistency across reviewers and study sections. I would like (in my dreams) to see more coherency and competency in the NIH agenda but 1) eyes of beholder and 2) too many goals and possibly 3) feature not a bug. I don’t particularly like the way demographics have skewed the workforce in our industry-in this I blame the Boomers. I fret over what has happened to a key generation of scientists in the wake of the BabyBoom . I am also really not keen on the way the GlamourMagification of science leads to economic inefficiency- keeps data from being published that would keep three more saps from having to retread the same (negative) ground. Not keen on the way it heads off replication and extension across multiple labs, either.
Word is that the former head of MSKCC (and incidentally the NIH) is going to co-sign the letter. That and $2.25 on your metrocard will get you downtown where the money is…
The frau was on the initial list and she told me of this letter. I replied that Collins will do what Congress tells him to do, not what the scientists say. And since scientists are mostly an apolitical lot (with a good number of them ineligible to vote being, like the frau, a foreign citizen) they have zero chance of having any influence.
If they want to save money, they should start with closing NCCAM & NIDA. It appears they only funds cranks anyway.
For Pete’s sake. As someone who can remember the days of unlimited resubmissions (as well as lots of money), it was excruciating. One really did try to figure out What The Damn Study Section Wanted. As an asst prof I tweaked one grant three or four times before I gave up. I’ve been funded and not funded, while being funded is better, writing endless revisions is truly awful. It does not make the creative juices flow.
I mentor a lot of young faculty. I see a lot of frustration and anger. Adding extra resubmissions isn’t going to help anyone in particular. Taking a good class on how to write a proposal and getting that A1 to a colleague is more likely to help.
I’d like to see a mechanism for the first timers, but at least in the two IC’s I work with, the payline for K-awards hasn’t been cut as badly as for R01’s. In my dept, we try to use higher level K’s (23) as a first 5-year R01.
Finally, I think a good case could be made for only one submission. You’re judged on what you want to do, not how well you address what the study section thinks.
Pushing A0s and A1 back a cycle for A2s that are equally meritoroius (beyond the limits of resolution in many cases) makes arithmetic sense when the funding levels are as low as they are now. If funding levels improve even slightly (if they get worse ALL bets are off) many people that you advocate eliminating from the system will be still be around and contributing great science.
By young investigators I was referring to those beyond the grace period, and will make this clearer in the formal petition.
Ahh, the good Professor Benezra himself. So I’ll take the opportunity to ask the key question here.
My question for you is quite simple: “Where were you?”
Where were you when the policies that systematically disadvantaged young/ESI investigators were being established and locked into stone? You refer to study section service. I wonder if you were the burr under the saddle of study section culture, constantly advocating for the ESIs, debating the merits of “Well Prof BlueHair is so damn productive that we gotta give this clusterbork of a proposal a good score anyway”, dismantling the expectation for yet more preliminary data and prioritizing giant, GlamourMag chasing labs over those of newbs. Were you?
When the NIH started making noises about special paylines for ESI investigators….did you endorse them enthusiastically? Did you use these moves during discussion at Study Section to further emphasize your position vis a vis helping the newbies? Did you rebut your peers who were grumbling? And how about the great Assistant Professor Purge from study sections? Did you cheer Scarpa along or did you oppose that?
Where were you on new hires, for that matter, if you have been in a position to affect such matters? Were you one who opposed trends for more and more postdoctoral training? Or did you slot right in a nod sagely with the wagging gray beards (From the Googs, actually I think Prof Benezra came along far too late to have a significant effect on the *start* of this process)?
And to be clear, this goes beyond the personal situation of the petition writer himself. I generalize this to all of the experienced and comfortably funded (until recently) PIs who are signing the petition.
I find the timing to be highly suspicious. I suspect that this crying over the fate of ESI under the A1 and out rules to be disingenuous. I suspect quite strongly (and to be honest, this is from talking to more than one established, continually well funded investigator much like Professor Benezra, not just random internet clownery) that what is really motivating people is the fear about their own grant sinecures. Their own expectation of continual competing renewals of their core projects. Their outrage that finally, finally, at long last, they themselves are facing the uncertainties and fears of losing grants that many folks of my approximate transition generation and below have had as a reality for our entire careers.
To get me fully on board with your proposals, my friends who are fond of this #BringBacktheA2 effort, you are going to have to show me that this is not, yet again, about naked self interest on the part of experienced investigators.
Ping!
DM- Just because a senior investigator is now concerned about their own funding situation does not mean that they are not capable of extrapolating what the current funding situation/rules might mean for some one less junior, or what the consequences might be for the publicly funded scientific enterprise in this country in general, and being genuinely motivated to do something about it.
Where was I? Well, I was writing a boatload of grants and fighting to keep my lab.
Of course it doesn’t mean they are not capable. But as someone who has heard a decided lack of interest or sympathy for the plight of the ESI over the past decade, decade and a half from people in a similar situation to Benezra (and D. Noonan for that matter) I have very good reason to be suspicious of this sudden interest. Particularly when it is my belief that what is really going on here is that these established types are realizing that this loss of the A2 puts endless competing continuation funding at substantial risk.
And of course I know where your motivations are coming from drdrA and I have confidence it is a good place. That’s because you happen to blog about your opinions.
We do, however, have a longstanding disagreement over just how sorry we should be feeling for tentatively established investigators seeking their 2nd grant or first renewal. That is a matter of degree, I suspect.
Pingback: Crocodile tears from experienced NIH investigators over the discontinued A2 revision [DrugMonkey] | iPhone 2 die 4
Actually I was inspired by the Egyptian revolt.
yes but given your NIH career success and closeness to Varmus that makes you, what, one of the hangers-on of the military oligarchy rather than one of the dudes in Tahrir square, amirite?
Dr. Benezra, with respect, I just don’t understand your reasoning. Why not just fund all these A2’s when they are A0’s or A1’s? Same people, same ideas, same capabilities. Look at the stats Drug Monkey refers to – all those A2’s will simply move ahead a cycle. I am dubious that a second round of revision really improves the grant, much less the science that eventually comes out of the grant. Elimination of the A2 has zero relationship with the overall funding rates.
Pingback: A Sneak Real Into Natalie's Mind » Blog Archive » Crocodile tears from experienced NIH investigators over the discontinued A2 revision [DrugMonkey]
Nice work, drdrA, way to spark the conversation!
http://news.sciencemag.org/scienceinsider/2011/02/frustrated-researchers-to-petition.html
DM-
I know you are at a medical school, and you think about people at hard money positions differently. The fact of the matter is that going from 2 R01s to 1 R01 as many medical school people will be doing is not as bad as going from 1 R01 to no R01s but having hard money. Yes you can still exist because you have a salary, but with funding as tight as it is, many study sections appear to be going towards rules that once you lose funding, you will not get your funding back.
1) You lose renewal because you weren’t productive enough over the period, regardless of whether you accomplished your aims in fewer publications, or you hit a somewhat difficult patch (this is life)
2) You can’t get a new grant because you’ve lost funding and are not as productive in light of this, and the first tiebreaker for all of these great grants is productivity.
3) You are toast.
I just had word passed to me that ESI applications are now prioritized (in some study sections) by those that have an independent publication from their lab.
I’ve picked up a few comments from you (DM) that make it seem you favor some sort of war between researchers at medical schools and those at Tier 1 Universities in departments with undergrads. We are all in this together- and we are all f***ed if things stay this way for awhile.
Neurowoman, when funding drops to 10-12 percent- some high scoring grants are only distinguished from other grants based on amount of data, not how worthwhile the science is. I can pretty much guarantee that someone unfunded now on an A1 at 16th or 17th percentile, probably has an excellent grant, with criticism being very minor. A2 will allow these grants to stay in the system with one less level of bullsh*t to go through. This is why I favor a cutoff for allowing an A2. If you are in a particular gray area.
Where do you get the cockamamie idea that in order to write a new application you need different preliminary data?
Pinko Punko-
I appreciate your comment. Teaching 70 or 80 or more hours per year can have the advantage of a mostly hard money salary- but it has a distinct disadvantage when you do not get your renewal and you enter this downward spiral. When you lose your single R01- research in your lab comes to a complete halt, except what you can do with undergrads essentially for free. In that case you better hope you don’t work with expensive reagents (read experimental animals, cell culture etc).
This is distinctly different than going from 2 R01s to 1R01- where you can still afford to employ personnel that are generating preliminary data, doing this or that experiment to satisfy this and that new application.
At our faculty meeting somebody brought up a rumour inre a slightly different push for bringing back the A2: i.e. that folk with a score within x pts of the payline would be allowed this privilege. Anyone else heard that one? Of course, that will simply introduce a narrow window holding pattern, but maybe that’s
Maybe that could cut both ways. Get triaged on A0 and you don’t get to even submit the A1 (ducks for cover). Brutal, but it would chop uninspiring projects out of the loop immediately and/or punish crap grantsmanship. That might loosen up a bit of time and effort to put back A2s for the close-but-no-cigar applications.
Better, the NIH should just put the whole review process in the paws of that fat squirrel fella, Pocahontas Bill or whatever his name is.
“but maybe that’s …” still an slight improvement.
PP- I believe the point is to get that A2 funded when it’s an A0 or A1, not make it get in line and wait its turn.
I’ve picked up a few comments from you (DM) that make it seem you favor some sort of war between researchers at medical schools and those at Tier 1 Universities in departments with undergrads. We are all in this together- and we are all f***ed if things stay this way for awhile.
If you are referring to my occasional comment that people who are on hard money gigs are distracted by nonNIH responsibilities and therefore a waste of NIH dollars, understand this is said only as a counter to those that whinge about softmoney researchers and talk about capping federally supported salary at less than 100%.
drdrA- your fantasy sounds nice but how do you conveniently arrive at the 2 -> 1 as equivalent to the 1 -> 0? Softmoney folks can just as easily get down to zero or R21/R03 fumes. They are then out of a job. Not live to write another grant. Not preserve small amounts of space to putter. Done. You also neglect to consider that when one has to cover 100% of one’s salary on grant funding, that last R01 doesn’t go very far. Don’t get me wrong, we all made the bargains that we did in selecting hard- versus soft-money job categories and I don’t think anyone has too much call to call the waaahmbulance.
But don’t try to pretend that proposed solutions like “one R01 for all” have the same impact on everyone. These solutions are in the favor of hard-money job categories and at the expense of soft-money categories. Fine to cry “Do it to Julia…not me, JULIA”, just don’t try to pretend there’s no self-interest at work here.
DM-
Once funding is below a critical level, then the question is where to make the dollars go. There will be hard questions. Some of us are on hard money, some are on soft. The reality is that NIH didn’t do enough to limit the proliferation of soft money positions with the doubling of the budget. In my estimation, hard money positions have at least hypothetically some natural limit to the existence, so they should be anchored to something in the system structurally, like tuition dollars, or number of undergraduates or what not. If one category has exploded over the other, this is an issue. One way to make grants go further is have them less tied to investigator salaries and have a greater institutional commitment on that side. This is an incredibly hard question. I don’t take this side because I am on hard salary (many people won’t have a choice, they take a job where they get a job). I have family who are tenured faculty at medical schools, did my training in medical school graduate program, and post-docs in medical school located labs.
At all of these institutions there were hard feelings between undergrad departments and med school departments. Who had the newer, nicer, larger lab spaces? Where were much or likely most of the NIH doubling dollars spent? We will all be feeling the severe pain. I think it will not be evenly distributed between hard and soft positions, I think there will be likely a greater correction onto soft money types of positions if (IF) that is where the majority of the bubble was.
Neurowoman,
I know what you mean. My comment relates to when funding is in the 12th percentile. This is not realistic to fund applications that are generally excellent. I think between 10-30 percentile the dings are either arbitrary or very minor, or the decision starts to go towards tiebreakers that are not really the best for determining meritorious proposals, such as philosophical considerations on how to fund crystallographic experiments (only when they’re done!) or if some PIs “environment” is a 2 and not a 1, when 1 is reserved for world-class institutions.
Pinko Punko, I think we pretty much agree that there are no easy and trite solutions here. But when you say
One way to make grants go further is have them less tied to investigator salaries and have a greater institutional commitment on that side.
as if you actually believe this to be an obvious truth, it triggers the kinds of cracks I usually make. The bottom line is that to do this you need to either have philanthropy or some other tasks the Professor has to do to make up for the hard money salary. Let us say that is teaching undergraduates. Teaching is not an easy job, particularly if you want to do it well. It takes time and reflection and work. Brain power and daylight hours that are not available to the ongoing research. Let us recall that we all realize that scientists are not just working their 100% time in 40 neat hours per week. We are having thoughts and connections and analyses going on all the time, often at random times of the day. When the NIH commands 50% of the run time of the soft-money brain, it gets full value. Even if the PI is shortshrifting the one project, that’s because she’s thinking about the other NIH funded projects.
When an R01 is charged 50% of a teaching professor’s brain run time, it isn’t really getting the same attention because other tasks are in there interfering.
I’m open to the obvious counter that a teaching prof of some experience can basically just phone in his lectures on 5% actual brain effort…so in that case the NIH *is* getting a good deal.
Physioprof- Some kinds of preliminary data lend themselves to multiple directions better than other kinds of preliminary data. I’m sure we will argue about that. The kind of preliminary data I generate, for example- lends itself extraordinarily well to spinning off multiple projects. Folks with a more hypothesis driven approach generate more specific preliminary data- harder to adapt to multiple angles on the same problem. My experience is that reviewers aren’t looking for simple feasibility anymore- they are looking for pieces of data in support of specific hypotheses. How do you use this kind of data in multiple grant submissions?
DM,
I know what you mean, but 100% soft money positions are not tenable. It is unclear what the norm is, perhaps somewhere between 50-75% it seems to me. Institutions somehow fund massive, shiny, wonderful new buildings. Buildings with novel features like house vacuum (inside joke between myself and a particular building).
Trust the EFF out of me, I would like to be spending all of my time in the lab. However, I do recognize that my experiences in research can open my undergraduates eyes about a lot of aspects of what they are learning, and I can challenge them, meaning there is a bonus for them in having a practicing scientist as professor. In turn, there is the possibility, ever so slight, that the raw material for new sausage in the eventual post-doc grinder will have received some sort of quality scientific priming at a decent undergrad institution, perhaps through profs funded by NIH or NSF. This means that the yahoos you will eventually see in your lab may not suck quite as much as they could. Given this contribution to society I conclude you should graciously walk the plank before myself. Do I have to type “/CLOSEJOKETAG”?
Why is it “not tenable”? How do you come up with your “norm” and why is that so superior? Seriously. What is “not tenable” is the number of profs three quarters distracted nursemaiding undergrads who don’t care for anything other than how to skate by on the least effort so they can work on their “American Idol” career or Myspace page…
You don’t seem to grasp the essential lesson taught by your parents- money doesn’t grow on trees. It has to come from somewhere. There is also no such thing as a free lunch- if you have hard money responsibilities, that’s effort that isn’t going into thinking about research.
You are talking to the wrong person about undergraduate inspiration. I was inspired by a set of teaching profs that had not done research in decades and I was absolutely *horrified* when I got to grad school to see what passed for undergraduate instruction at a major research Uni. No sale on that one. Having hard money alleged “teaching” profs 50% time on NIH funded research isn’t good for the teaching *either*.