Yesterday I cleaned my office. I’m now totally organized (insert *big* laugh here), and bare desktop is actually visible. Today, I tackled my mailbox. I’m following the rule that I don’t clean my mailbox out but every two weeks. I find that nothing in there is really that important anyway, if it is urgent people tend to send email… and call if totally desperate.
Anyway. I get this little magazine called ‘Microbe’ which is a publication of the American Society for Microbiology. I usually leaf through that thing briefly when it comes- and this morning I opened one that I found during the mailbox cleaning and came across a short guest commentary entitled ‘Descriptive Science’ written by Casadevall & Fang. Because I’ve had a front row seat the the effect of the technological revolution in microbiology, and I see a bias toward studying a small handful of factors that we already know about because we can make hypotheses about them… as opposed to scratching the surface on a vast trough of new knowledge- I care quite a lot about this topic. I’ll start by quoting directly and hopefully it will become clear where I’m going with this post:
The Instructions to Authors for Infection and Immunity state that “IAI will not consider papers that are… purely descriptive.” When applied to science, the word “descriptive’ has acquired dismissive or pejorative connotations and is frequently provided as justification for rejection of a manuscript or grant application.
Ah, yes. That second part is a big pet peeve of mine. I really, really, really dislike the fact that the word ‘descriptive’ in this context has turned into such a put-down. Why do I hate this? I guess I see this observational or descriptive aspect as integral to doing good science. After all, you can’t make a hypothesis if you don’t first make an observation (or follow up observations reported in the literature)- which is a purely descriptive finding. Observation is the mother of the hypothesis, if you aren’t good at this ‘descriptive’ part of science- doing that second part … the one with the hypothesis…. is going to be tough.
The authors go on to define the word ‘descriptive’ and decide that nearly all science is in one way or another ‘descriptive,’ and then say the following:
scientists distinguish between ‘descriptive research,’ in which information is collected without a particular question in mind, and ‘hypothesis-driven research,’ designed to test a specific explanation for a phenomenon. In this dichotomy, ‘descriptive’ has numerous synonyms, including ‘observational’, ‘inductive,’ or ‘fishing expedition’ while ‘hypothesis driven’ may also be referred to as ‘hypothetico-deductive’ or ‘mechanistic’. When scientists favor hypothesis driven science over descriptive science, they are really saying that they prefer work that is explanatory or provides insight into causation.
Uh huh, ok…And…
In microbiology, and related medical sciences, the transition from descriptive research to hypothesis-driven research has generally reflected the maturation of these fields. In the early stages of a field, descriptive studies may ‘represent the first scientific toe in the water’ (Grimes and Schulz 2002, Lancet 359).
But wait….Casadevall & Fang go on:
Microbiology and immunology are presently being transformed by a number of powerful technological advances; methods such as large-scale sequencing, microarrays, bioinformatics, and proteomics are generating enormous data-bases that provide invaluable resources for the research community. While these methods can certainly provide potent means to answer mechanistic hypotheses, in many cases they are initially being used solely in a ‘descriptive’ sense. In other words, some aspects of biological science have returned to an observational phase, in which research is primarily ‘discovery driven’ rather than ‘hypothesis driven’ (Aebersold 2000 Nature Biotechnology 18) .
Yes, microbiology and immunology are unquestionably being transformed, and this transformation is nothing short of revolutionary. I’m thrilled to be a microbiologist living and working in this genomic era. I’m fascinated by the mountain (or mountain range) of new data that I now have access to- and am in some small part participating in generating. For everything I think I know about microbiology, the new knowledge of each passing day is humbling in it’s scale, it’s complexity, and it’s shear volume.
I don’t think we ever left that ‘scientific toe in the water phase’ in this field. We may have had the illusion that our understanding was becoming more ‘mature,’, but I’m just not sure that this idea is particularly accurate. So- with that in mind- if we have returned to an ‘observational’ phase (or never left it in the first place)- what does that say or mean about what kind of data we can and should publish? Is it a good thing for our bias toward ‘hypothesis driven’ or mechanistic research- which can only even address a tiny fraction of the questions we have right now, be favored over all else? Where do the mountains of observational data go? Do they get published as a mountain with a single small part in each paper that has a mechanistic follow up (that kind of feels like window dressing most of the time)?
Are we going to have to adjust our idea of the value of descriptive science?
Blue Lab Coats 
Oh, fuck me! Not this crap again!! “Hypothesis-driven science” is a bunch of fucking assholes standing around under a streetlight ignoring the darkness that surrounds them.
There was a post back on DM’s blog in which some of the comments touched on the issue of descriptive science.
The article is quite painful to read.
“scientists distinguish between ‘descriptive research,’ in which information is collected without a particular question in mind, and ‘hypothesis-driven research,’ designed to test a specific explanation for a phenomenon.”
They most certainly do not if they are in any sense professional. Descriptive research is part and parcel of the hypothetic-deductive model, they are not separate models. The authors should really familiarise themselves with contemporary epistemology.
The problem of publishing descriptive science alone is that, sans any testable prediction, any conclusions derived from it are, as they state, purely inductive. The discourse on the pitfalls of relying on inductive reasoning are extensive from Hume to Popper to Homer Simpson. There are also pitfalls to the falsifiable hypothesis strategy, but it’s still a damn sight better than the alternatives.
To rely on observation without prediction, and to attempt to draw inferences from those observations, is to be highly vulnerable to a whole gamut of logical fallacies that can potentially lead the investigator and the field she’s working in astray. And don’t even get me started on people who insist on applying parametric statistics to data generated in this manner.
“methods such as large-scale sequencing, microarrays, bioinformatics, and proteomics are generating enormous data-bases that provide invaluable resources for the research community. While these methods can certainly provide potent means to answer mechanistic hypotheses, in many cases they are initially being used solely in a ‘descriptive’ sense.”
Bollocks. These techniques are supposed to be used to isolate observable phenomena for further investigation. They are supposed to represent the first stage in what should be a hypothesis-driven process. The latter part of that process being vital to weed out chance occurrences that are inevitably going to arise if one is assaying a large number of variable at any one time.
The accusation that pedantic scientists don’t like “descriptive science” is a straw man, because we all use it. You just don’t stop the project there.
‘ “Hypothesis-driven science” is a bunch of fucking assholes standing around under a streetlight …’
I’m glad WE agree. Now how to deal with study section, paper review etc. etc. etc….
Amen!
Converting back and forth from *thinking* “I wonder what happens if I do this…” and *writing grants about* “we hypothesize blah-de-blah depends on blah-blah-bleck” is a waste of mental energy better spent designing better dohickies, re-analyzing other people’s fish, or even (dare I say) ranting on blogs.
the entirety of science is a description of what happened under certain conditions. Period. It is ALL descriptive. The idea that one is “getting at mechanism” is a silly conceit pure and simple. Those that parrot this nonsense may really have no idea what they are doing, it strikes me.
For me this business of decrying what is essentially the heart of science (observing, describing and then producing hypothesis) is very worrying too. As many of us in microbiology are now doing large scale analysis it is imperative that they are properly reviewed and publishable in respectable journals otherwise why bother doing them? No publications = no grant = no job for many of us. The masses of data generated in such experiments often do start from a hypothesis, even if it is not confirmed. But what is most important is that such descriptions are published and available to all in the case that someone else might find something important in there that will inform their own observations.
While these methods can certainly provide potent means to answer mechanistic hypotheses, in many cases they are initially being used solely in a ‘descriptive’ sense. In other words, some aspects of biological science have returned to an observational phase, in which research is primarily ‘discovery driven’ rather than ‘hypothesis driven’
And what is so wrong with that? A lot of more recent microbiology is akin to trying to figure out how a very complex machine in a covered perspex glove box works by breaking individual parts of it and seeing how the machine still functions from the outside with the cover still on the box. Yes, absolutely this means that many variables can be controlled, but as someone who works at the interface of basic and clinical microbiology, I am acutely aware of the limitations of this approach.
To apply some of these newer ‘omics techniques means, in essence we can take the cover off the box and see the machine as whole. Yes we don’t fully understand it, but we have a much better idea what parts are in there and how under certain circumstances they might interact. We have much to learn by describing the mechanistic function of the organisms we are studying, maybe we do need to take a step back and use the very powerful techniques now at our disposal to get a more holistic understanding of the organisms we work on. The more we look at these ‘descriptive’ ‘discovery’ sets of data the more we usually see that there is more (often much more) to the picture we are looking at, these are in fact essential.
TOTALLY dumb. I run into this all the time, I have done some proteomics projects and for some reason it’s always some kind of apparent dumb-person taboo to do these “fishing expeditions.” Well how the fuck are all you fancy pants HYPOTHETICAL scientists (and me) supposed to get anything to HYPOTHESIZE about without some fishing expeditions? And what if the information dredged up is probably (no, surely) useful to other people?
Duh guys, really: duh.
This totally pisses me off. I’m about to resubmit a grant that proposes to look at a relatively novel cell signaling mechanism that I suspect is in overdrive in Chronic Disease and might go some way towards explaining the increased morbidity and mortality in patients with this disease but the first time it was reviewed, I was told it was “too descriptive”. How the fuck are we supposed to progress to hypothesis-driven bullshit unless you have clearly demonstrated that your hypothesis will hold water? Oh wait, I forgot … that’s supposed to be Preliminary Data. Proving that this mechanism is upregulated and alters cell signaling in the cell/organ/disease of interest is a 2yr study in and of itself. Grrrr.
I am glad that there are PI’s that are fighting this mechanistic mentality. In my old PhD lab, the awesome senor grad student was not able to get a good chunk of her thesis published because she couldn’t show the mechanism behind the observations. That is until she submitted to PLOS.
I am glad that there are PI’s that are fighting this mechanistic mentality.
Unfortunately, these PI’s are going to lose this fight badly when the NIH study sections triage their grants.
Mechanistic studies: I’m going to do what any of us could do, but just haven’t got around to yet.
Descriptive studies: I’m going to see something that no-one else has ever seen.
In the current risk-averse funding environment, if you want to maximize the chance of “successful studies”, mechanistic is the way to go.
Additionally, there is an implicit problem with proposing descriptive studies because the implication to the study sections is that they didn’t see this new thing themselves because they weren’t clever enough to figure out how to see it. Remember that the people controlling the cash are (nominally) your peers. It’s one thing to say to them, any of us could do this mechanistic study if we wanted to, but I’m getting to it first because the rest of your are busy doing other worthy things, or I have this special reagent, or this niche technique I can use. It’s a completely different thing to say to them, this important observation has been missed by all of you, because you weren’t smart enough to figure out how to find it, but I will now be able to do what you could not, so pay me to be smarter than you.
DSKS- Bollocks. These techniques are supposed to be used to isolate observable phenomena for further investigation.
I with you 100%. But the problem is as Dr. J. points out- takes $$ to do the large scale ‘descriptive’ work- takes papers to get the $$s, if you can’t publish it- you can’t get $$s for it and to me, this hampers the progress in science.
Dr. J.- I think your comment is spot on.
To apply some of these newer ‘omics techniques means, in essence we can take the cover off the box and see the machine as whole. Yes we don’t fully understand it, but we have a much better idea what parts are in there and how under certain circumstances they might interact. We have much to learn by describing the mechanistic function of the organisms we are studying, maybe we do need to take a step back and use the very powerful techniques now at our disposal to get a more holistic understanding of the organisms we work on.
This is my daily frustration.
Drugmonkey- I’m in total agreement with you- but you, and I , and Whimple (probably everyone else who reads this blog) know that the $$ follow the mechanistic only- and this situation is actively maintained- except in certain realms like $$ for sequencing projects- I’m not sure how you sell those as hypothesis driven- but they get paid for.
Whimple- I know this is the game and it has to be played, but I don’t have to like it or think it is right in the bigger scientific picture. And I don’t.
It’s a completely different thing to say to them, this important observation has been missed by all of you, because you weren’t smart enough to figure out how to find it, but I will now be able to do what you could not, so pay me to be smarter than you.
And as for this. I HATE that this gets turned into being about colleagues’ egos. When I look at someone else’s work and it is really beautiful, I think wow, that’s really beautiful, and not hey look at this guy and his uppity science trying to show me what an idiot I am. But I take your point in that there is a way to put things and a way not to put things… and there is a way to choose to use the way that is going to get you what you want.
And what if the information dredged up is probably (no, surely) useful to other people?
Well, if it’s surely useful to others, surely you can pull out an example or two to append to the end of the paper yourself! C’mon, is there any mystery why editors and reviewers prefer a data dump with at least a half-baked example of its utility over “Trust me, it’s important, but it’s someone else’s job to use it!”? Even the human freaking genome had to jump through that hoop, never mind your jumble of 2D gels.
Get money to perform “mechanistic” studies that are already 95% complete, and then use it to do the really interesting shit.
Uch, I totally agree. How can we come up with hypotheses if we don’t go out and observe patterns in the first place? In some cases (take climate change) there is no way in hell to conduct an experiment. So we just don’t study it, right? Duh.
I think correlational and descriptive research is of immense importance, however, the reason why this might have such a negative taste to many may be that even though such data is easy to gather (relatively to experimental data, because one doesn’t necessarily need to think prior data collection), but it takes a good scientist to draw conclusions of such data. It happens quite often that people mistake correlation for causation.
On the other hand, many experiments are also just bullshit, makes me cry when I see how some people treat their data, which costs $$$ to generate and then draw the wrong conclusions. Or experiments with N=2, ouch. I think both “methods” if one could separate them, have their pros and pitfalls.
C PP- Yeah. I got that. It’s probably worth a whole post that you write the mechanistic, and do what you really want to once the $$ are in hand. But it SUCKS that it has to be this way.
Whimple you are missing the point that some of these people telling hypothesis zombies they are full of shit are actually on study sections. impressionable future study section members may be reading academic blogs. overheated rhetoric for support (me) and rational bullet points (everyone else) to deploy will help to stiffen the spine of n00b study section members.
there IS a goal here.
(oh and for anyone who simply cannot avoid putting their fishing expedition into a proposal make sure to use the phrase “hypothesis generating aspect of the plan”.)
DM- In fact just this morning I had a conversation with a colleague who said that this exact issue came up in a recent study section discussion that this person was involved in. I think that there are people (read study section members) that feel uncomfortable with the ‘descriptive’ moniker as a put-down in study section, but there is relatively little active standing up for this kind of work. It’s a shame. If discussions like this one change some minds, and give people a list of rational points to deploy when the discussion comes up in study section… well, then I’ve done my job here.
And thanks for recognizing that this IS the goal.
DM, you have to be a realist about these things. Actually, I think the proper way to go is for the NIH to abandon R01s to the mechanistic people and institute a new granting system for descriptive studies. There is already a little movement in this direction with the Pioneer, T-R01 and EUREKA mechanisms, and I’d like to see this expanded (at the expense of funding mechanistic R01s if necessary). NIH R01 study section behavior isn’t going to change fast enough to do anybody any good, no matter how forcefully we scream into the wind about it.
NIH R01 study section behavior isn’t going to change fast enough to do anybody any good, no matter how forcefully we scream into the wind about it.
I would personally beg to differ on that one. I have seen a shift over some time. I have seen grants get funded because of some of these shifts that would very likely not have been funded before these changes. Not that they know it, but there are a handful of PIs out there who would consider these shifts “good”.
Now it may be the case that we’ve just sort of shifted from the bad culture of one set of reviewers to the bad culture of another. But since the changes are in a direction I like, hahahhahaha!
I think DSKS put it best. The fishing should lead to a hypothesis. The only thing I’d add is that there’s a big difference between a good and bad fishing expedition. Good ones have an idea of a) where the fish are, and b) what kind of fish they’ll be. You should have an idea of what the hypotheses will be. If not, it’s merely descriptive science.
Microarrays are a good example. Anyone can propose to do a set of arrays, but only a small fraction will produce useful information. An array should not be employed “solely in the descriptive sense.”
To piggy back off of what DM said, while it’s good to insert the “hypothesis generating” clause, but you better spend a lot of time explaining the hypotheses you expect to generate.
Just think about what quality discoveries and pathways may not have been discovered if the pertinent scientists didn’t go on so called fishing experiments or blue sky ideas. I agree that if I have an idea of what I will find – why not? If it can be done on the smell of an oily rag – why not?
DM said,
“the entirety of science is a description of what happened under certain conditions.”
And in doing so highlighted the fact that semantics might be generating a storm in a teacup on this issue. I’ve seen that definition used a fair bit, and it’s also common to see the slightly narrower definition of descriptive science as being “knowledge for its own sake” vs normative science (an issue that has been well handled with the formation of the NIH on the one hand and the NSF on the other).
Then we have the authors of the linked article in the original post, and the editors of IAI, who seem to be associating “descriptive science” with initial observations (hypothesis-free), creating a fyrther definition. If the authors are arguing that the first step in the hypothetico-deductive model should be allowed to stand on its own and be published, then there are well established caveats to doing so (see Black Sheep joke). On this note, I’d actually agree with the other commenter who suggested a separate journal for this kind of thing, because although sharing initial observations could be very useful for the scientific community, it would be important to separate purely inductive research from conventional hypothesis-driven research.
Regarding fishing expeditions (omics etc), the term “discovery-based science” has been used. It’s important not to conflate that with “initial observations” though because, as Steven Wiley wrote in The Scientist even fishing-expeditions are generally hypothesis-driven. The difference is usually only in the scope and strength of the hypothesis.
So in response to DM’s comment,
“(oh and for anyone who simply cannot avoid putting their fishing expedition into a proposal make sure to use the phrase “hypothesis generating aspect of the plan”.)”
You can give a stronger argument than that. The fishing expedition is not only hypothesis-generating, but is motivated by a hypothesis in the first place. Professor in Training states his hypothesis for the “descriptive” project he was accused of proposing very clearly in his comment. Of course all new and exciting avenues of research necessarily originate from a very broad question, which often requires casting out a wide net. It’s either that or let science rest on some pretty spectacular leaps of intuition to get it directly from Disease X to a frame-shift generated stop codon in the middle of the mRNA transcript for gene Y. If the preliminary data is compelling enough to apply strength to that broad hypothesis (i.e. indicate that, as Matthew describes, this particular fishing expedition might be worth the time and money), one can easily argue that the outcome of the study will facilitate the progressively narrower generation of subsequent hypotheses, which will ultimately begin to isolate the most probable mechanism.
And another thing…!
Ok, now I think I have no idea what any of this means.
Is the difference between a descriptive hypothesis and a mechanistic hypothesis that the mechanistic hypothesis is falsifiable?
Is this philosophically like difference between analog and digital? Descriptive science means you measure some physical parameter; you can get any value. Mechanistic science means you support or refute a hypothesis; you can only get values of true or false?
I’m struggling with a grant re-application and desperately need help clarifying this point.
HELP!
Whimple said,
“Is the difference between a descriptive hypothesis and a mechanistic hypothesis that the mechanistic hypothesis is falsifiable?”
A hypothesis must be falsifiable period, whether broad and weakly-focused (omics, “fishing expeditions” and so forth) or narrow and explicit. I think the latter is what is often referred to as “mechanistic” science, even though all science is ultimately directed towards establishing cause.
I think the current brouhaha about “descriptive science” conflates two separate issues best described by the following examples:
1) The serendipitous discovery: The investigator is testing her prediction that Treatment A will increase Variable X. During analysis, however, she discovers a trend indicating that Treatment A also appears to have changed Variable Y.
2) The fishing expedition: The investigator enters a fledgling research field regarding a new disease for which little is known of the cause. He proposes to do a genetic screen to see whether the disease is simply a result of a genetic defect. He finds that expression of genes X, Y and Z is deficient in all patients with the disease, but no patients in the control group.
(1) is hypothesis-free and (2) is hypothesis-driven. Both are clearly hypothesis-generating. Under current standards, (1) cannot be published without at least designing new experiments to test the hypothesis that Treatment A affects Variable Y (only on this subsequent data can statistics be applied; assessments of probability post hoc are invalid). (2) Satisfies the hypothetico-deductive model completely, and the issue determining publication will be the impact of the work and the reliability of the methodology.
Pingback: Descriptive vs. Hypothesis-driven, part II « Blue Lab Coats
“Descriptive Science Blue Lab Coats” was indeed a great
blog. However, if it included much more pictures it would certainly be perhaps even far better.
Cya -Marta