Do you have a fitness tracker? Are you on Twitter or Facebook and count your likes and followers? Do you know your ResearchGate score? Do you pay attention to Gault Millau toques and Michelin stars when you visit restaurants? Then you’re in good company, because you’re doing reputation management on a wide variety of levels with quantitative indicators. Just like universities and research sponsors. Except that you do it privately and entirely voluntarily!
On these pages I have recently discussed (here, and here) why in academia today we hardly judge research on the basis of its originality, quality and true scientific or societal impact. Instead, we use quantitative indicators such as Journal Impact Factor (JIF) or third-party funding, and distribute grants or academic titles based on these indicators. I also pondered a few foolish ideas on how to turn the wheel back a bit, in the direction of a content-based evaluation of research achievements. But these considerations still failed to take into account that institutions and funding agencies are in good company – namely ours – when they foster competition with simple, abstract metrics. This makes things easier for them. And at the same time, harder for us to change the system. Because we may have to change ourselves.Continue reading
Science gobbles up massive amounts of societal resources, not just financial ones. For academic research in particular, which is self-governing and likes to invoke the freedom of research (which in Germany is even enshrined in the constitution), this raises the question of how it allocates the resources made available to it by society. There is no natural limit to how much research can be done – but there is certainly a limit to the resources that society can and will allocate to research. So which research should be funded, which scientists should be supported?
Mechanisms for answering these questions, which are central to the academic enterprise, have evolved evolutionarily over many decades. However, these mechanisms control not only the distribution of funds among researchers and within as well as between institutions, but ultimately also the content and quality of research. The mechanisms by which research funds and tenure are evaluated and allocated and the metrics used in these processes determine scientists’ daily routines and the way they do research more than their reading literature, their views through a microscope, or their presentations at conferences. Continue reading
Despite the fact that the number of cases is now rising sharply again and we have now entered a lockdown ‘light’, we in Germany are rightly pleased that we have so far come through the corona crisis much better than many of our neighbors or the USA. Was the ‘German way’ perhaps so successful because politicians in Germany had an open ear for science and therefore prescribed the right measures based on evidence?
(Update August 2020)
On March 17th, just as many countries were taking draconian measures to contain the SARS-COV-2 pandemic, the Greek-American meta-researcher and epidemiologist John Ioannidis, whom I often quote in my posts proclaimed a “fiasco in the making‘! With strong language and a few ad hoc estimations of COVID fatality rates he warned that based on poor data or no evidence at all politicians might inflict incalculable damage on society, possibly much worse than what a virus, putatatively as dangerous as influenza, could cause. As one of the most highly cited researchers in the world and a vocal critic of quality problems in biomedicine, his COVID related interviews, opinion pieces and articles since then have received a great deal of attention, in the scientific community, in the lay press, and especially among his worldwide fan base. Continue reading
Research using animals is a sensitive issue. Anyone who does animal experiments, like myself, is reluctant to talk about it, at least outside our natural habitat, the laboratory or scientific conventions. Institutions where animal experiments are carried out are also quite shy about the topic. Recently, the Max Planck Society left Nikos Logothetis (MPI Tübingen) standing in the rain when he was targeted by a media campaign. Now he and some of his laboratory staff are off to Shanghai… The websites of prominent research institutes feature all kinds of colorful illustrations showing immunohistochemistry slides, doctors and students in white coats with pipettes in their hands, sitting at computers or their microscopes. But rats or mice are conspicuously missing! They proudly display their research activities, and enthusiastically advertise (future) research breakthroughs towards completely new and effective therapies. But no reference is made to animal experiments on campus!
Anyone who follows this blog must get the impression that I am a nag and misanthropist. Nothing and nobody seems to please me. I alway find the sample sizes too small, the statistics too lazy, the data hand-picked, or the results too positive and the conclusions drawn from them too exaggerated. Also, I find the peer review system unreliable, not to mention support of mainstream research and giving to those who already have (‘Matthew effec’) by the funding agencies. I even dared to critsize the Nobel Prize an atavistic instrument celebrating the lonely, white male reseacher and genius. Artificial intelligence I find stupid, and the academic career system the core of all these evils. To name but just a few examples.
But that is far from the truth! I am a science enthusiast! I am convinced that science is the best that the 1500 grams of protein and fat encapsulated in our skull have ever produced. Yes, I am a science nut. So with this entry, at the beginning of the new decade, let’s start with a proper hymn to biomedical science. Continue reading
You got to see this youtube video! Hectically cut sequences of busy young scientists in high-tech laboratories wearing lab coats, nerdy looking guys are soldering electronic circuits and stare into oscilloscopes, we are taken on a roller coaster ride through an animated brain chockful of tangled nerve cells. And in between all this, on stage at the California Academy of Sciences, car and rocket manufacturer Elon Musk announces his latest vision in a messianic pose: The symbiosis of the human brain with artificial intelligence (AI)! This time his plan to save mankind does not involve mass evacuation to Mars, but will be realized by a revolutionary Brain Machine Interface (BMI), designed and manufactured by his company Neuralink. You may have guessed it, this has caused a tremendous media hype all over the world. The verdict in the press and on the net was: “Musk at his best, a bit over the edge, but if HE announces a breakthrough like that there must be something to it”. The more cautious asked: “But couldn’t this be dangerous for mankind? Do we need a new ethic for stuff like this?” Continue reading
Damn! What an effort: Generation of a knockout mouse line, back crossing in background strain and litermates, all the genotyping. Followed by a plethora of experiments in a disease model: surgery, magnetic resonance imaging, histology, behavioral studies, and so on. Finally the result: No phenotype! The knockout mouse appears to be a mouse like any other. Not different from the wild type background strain. But wait, we rather need to phrase it like this: We did not find a statistically significant difference between knockout and wild type. So we cannot even conclude that wild type are like knowout mice, but rather: If there is a difference, it might be smaller than the detectable effect size, depended on sample size, error level (alpha and beta) and the variance of our results. But we had planned our experiments well: The sample size was determined a priori, and chosen so that we would have been able to detect a difference on the order of one standard deviation. This is what statisticians call a Cohen’s d of 1, which is considered a substantial effect. We could not have done more animals than the (34!), because of limited ressources, the duration of the PhD thesis, and the timing of the grant. But what now? Write a paper? Reporting a NULL result? How would this look like in a resume, besides, who cares about NULL results, and which reputable journal would publish them at all? Continue reading