Einsichten eines Wissenschaftsnarren

narrLeseempfehlungen zur Laborjournal Kolumne:

(Und hier geht es zum Archiv aller Artikel)




Einsichten eines Wissenschaftsnarren (10):

Gelman A and Loken E (2013) The garden of forking paths: Why multiple comparisons can be a problem, even when there is no  “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf


Gelman A and Loken E (2014) The statistical crisis in science. American Scientist 102:460-465


de Groot AD (1956) The meaning of “significance” for different types of research [translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der Maas]. Acta Psychol (Amst). 2014 May;148:188-94. doi: 10.1016/j.actpsy.2014.02.001. Epub 2014 Mar 3. http://www.sciencedirect.com/science/article/pii/S0001691814000304?via%3Dihub


Yogi Berra: When you come to a fork in the road, take it! https://quoteinvestigator.com/2013/07/25/fork-road/



Einsichten eines Wissenschaftsnarren (9):

Researchers were disappointed when a clinical trial of a new tuberculosis vaccine failed to show benefit, but should it have gone ahead when animal studies had already raised doubts and what does it mean for future research? Deborah Cohen investigates:

Cohen D. Oxford vaccine study highlights pick and mix approach to preclinical research. BMJ. 2018 Jan 10;360:j5845. http://www.bmj.com/content/360/bmj.j5845.long


We must review how we use animal data to underpin clinical trials in humans:

Macleod M. Learning lessons from MVA85A, a failed booster vaccine for BCG. BMJ. 2018 Jan 10;360:k66. http://www.bmj.com/content/360/bmj.k66.long


Responsible bodies should demand high quality reporting and systematic reviews of animal studies:

Ritskes-Hoitinga M, Wever K.Improving the conduct, reporting, and appraisal of animal research. BMJ. 2018 Jan 10;360:j4935. http://www.bmj.com/content/360/bmj.j4935.long


A systematic review of the animal data does not provide evidence to support efficacy of MVA85A as a BCG booster:

Kashangura R, Sena ES, Young T, Garner P. Effects of MVA85A vaccine on tuberculosis challenge in animals: systematic review. Int J Epidemiol. 2015 Dec;44(6):1970-81.  https://academic.oup.com/ije/article/44/6/1970/2572503


Einsichten eines Wissenschaftsnarren (8):

As always, David Colquhoun brilliantly summarizes the issue in his blog DC’s Improbable Science:

Colquhoun D. Placebo effects are weak: regression to the mean is the main reason ineffective treatments appear to work. http://www.dcscience.net/2015/12/11/placebo-effects-are-weak-regression-to-the-mean-is-the-main-reason-ineffective-treatments-appear-to-work/


Highly recommended. Stephen Senn tackles the issue with a historical excursion, and a bit more serious statistics. Contains references to full statistical treatments of regression to the mean:

Senn S (2011)  Francis Galton and regression to the mean. Significance 8:124-126 http://onlinelibrary.wiley.com/doi/10.1111/j.1740-9713.2011.00509.x/full


When the first large Cochrane review was published, concluding that there is little evidence to support the existence of the placebo effect, the New England Journal ran this very readable editorial:

Bailar JC (2001) The Powerful Placebo and the Wizard of Oz. N Engl J Med 2001; 344:1630-1632 http://www.nejm.org/doi/full/10.1056/NEJM200105243442111


Most recent Cochrane review on placebo interventions:

Hróbjartsson A, Gøtzsche PC (2010)  Placebo interventions for all clinical conditions. Cochrane Database Syst Rev. 2010 Jan 20;(1):CD003974. doi: 10.1002/14651858.CD003974.pub3


This narrative review present the classic view on placebo effects. The authors use only one citation (to another narrative review by the same authors) to claim the existence of the placebo effect:

 ‘ Moreover, recent clinical research into placebo effects has provided compelling evidence that these effects are genuine biopsychosocial phenomena that represent more than simply spontaneous remission, normal symptom fluctuations, and regression to the mean. “

It would have been nice to see reference to evidence from original studies or systematic reviews.

Kaptchuk TJ, Miller FG (2015) Placebo Effects in Medicine. N Engl J Med 2015; 373:8-9July 2, 2015 http://www.nejm.org/doi/full/10.1056/NEJMp1504023#t=article



Einsichten eines Wissenschaftsnarren (7):

OPERA collaboration, T. Adam et al., Measurement of the neutrino velocity with the OPERA detector in the CNGS beam, arXiv:1109.4897v2.

Daniel J. Benjamin et al. Redefine statistical significance. Nat Hum Behav Published 1: 0189. DOI: 10.1038/s41562-017-0189-z

Valentin Amrhein, Sander Greenland. Remove, rather than redefine, statistical significance Nat human behaviour Nature Human Behaviour. 1: 0224. doi:10.1038/s41562-017-0224-0


Einsichten eines Wissenschaftsnarren (6):

Yong, E. The Absurdity of the Nobel Prizes in Science – The Atlantic

Knorr-Cetina, Karin D.: Epistemic Cultures – How the Sciences Make Knowledge. Cambridge: Harvard University Press, 1999


Einsichten eines Wissenschaftsnarren (5): 

Kimmelman J, Mogil JS, Dirnagl U. Distinguishing between exploratory and confirmatory preclinical research will improve translation. PLoS Biol. 2014 May 20;12(5):e1001863. doi: 10.1371/journal.pbio.1001863. eCollection 2014 May. http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1001863

Mogil JS, Macleod MR. No publication without confirmation. Nature. 2017 Feb 22;542(7642):409-11. doi: 10.1038/542409a. PMID: 28230138; https://www.nature.com/news/no-publication-without-confirmation-1.21509

Bazerman,  Shaping Written Knowledge: The Genre and Activity of the Experimental Article in Science: Genre and Activity of Experimental Article in Science, University of Wisconsin Press, 1988. https://wac.colostate.edu/books/bazerman_shaping/shaping.pdf


Einsichten eines Wissenschaftsnarren (4): Exzellenztheater: Zeit für einen Wechsel im Spielplan?

Der Titel sagt eigentlich alles: Fang, Ferric C., Anthony Bowen, and Arturo Casadevall. 2016. “NIH Peer Review Percentile Scores Are Poorly Predictive of Grant Productivity.” eLife 5 (February). doi: 10.7554/eLife.13323 . https://elifesciences.org/articles/13323

Ein Klassiker der Wissenschaftsgeschichte und -theorie. Nach TS Kuhn schreitet die Evolution von Wissen fort durch „normal science“, unterbrochen durch ‚Paradigmenwechsel‘ (da kommt auch der Begriff her): Kuhn TS. The structure of scientific revolutions. University of Chicago Press. 1962 (50th anniversary edition 2012)

Richard Münch’s 500 seitiger Frontalangriff auf Exzellenzinitiative, DFG, usw. Nichts für schwache Nerven: Münch, R. Die akademische Elite: Zur sozialen Konstruktion wissenschaftlicher Exzellenz (edition suhrkamp 2510). Frankfurt am Main 2007

Eine fundierte Abrechnung mit dem Begriff ‚Excellence‘, vor dem Hintergrund der englischen REF: Stilgoe, Jack. 2014. “Against Excellence.” The Guardian , December 19. https://www.theguardian.com/science/political-science/2014/dec/19/against-excellence


Einsichten eines Wissenschaftsnarren (3): Werden Sie Forschungsförderer!

Diese beiden Arbeiten beschreiben das System in allen Einzelheiten:

Bollen J, Crandall D, Junk D, Ding Y, Börner K. From funding agencies to scientific agency: Collective allocation of science funding as an alternative to peer review. EMBO Rep. 2014 Feb;15(2):131-3. http://embor.embopress.org/content/15/2/131

Diese Arbeit ‘erprobt’ das System zudem in einer Simulation: Bollen, J., Crandall, D., Junk, D. et al. An efficient system to fund science: from proposal review to peer-to-peer distributions Scientometrics (2017) 110: 521. doi:10.1007/s11192-016-2110-3  https://link.springer.com/article/10.1007%2Fs11192-016-2110-3

Das schreibt Retraction Watch zum peer funding: https://www.evernote.com/shard/s21/nl/2303872/2592c704-d95b-41b6-aba9-00b4565aeab6/ ..

… und dies Science: http://www.sciencemag.org/news/2017/04/new-system-scientists-never-have-write-grant-application-again


Und dies kam eben rein (19.9.2017)

How much would each researcher receive if competitive government research funding were distributed equally among researchers?

‘According to our results, researchers could, on average, maintain current PhD student and Postdoc employment levels, and still have at their disposal a moderate (the U.K.) to considerable (the Netherlands, U.S.) budget for travel and equipment. This suggests that the worry that egalitarian sharing leads to unacceptable dilution of resources is unjustified. Indeed, our results strongly suggest that there is room for far more egalitarian distribution of funds than happens in the highly competitive funding schemes so prevalent today.’

PLoS ONE 12(9): e0183967. https://doi.org/10.1371/journal.pone.0183967


Weitere interessante Arbeiten:

NIH Funding für den Mainstream, innovative Arbeiten werden nicht oder woanders gefördert:

Nicholson JM, Ioannidis JP. Research grants: Conform and be funded. Nature. 2012 Dec 6;492(7427):34-6. https://www.nature.com/nature/journal/v492/n7427/full/492034a.html

‘Peer review of grant proposals, far from being a reasonable way of ensuring quality by the allocation of funds, is a near disaster.’: Horrobin DF. Peer review of grant applications: a harbinger for mediocrity in clinical research? Lancet. 1996 Nov 9;348(9037):1293-5 http://www.sciencedirect.com/science/article/pii/S0140673696080294

Nicholson, J. M. (2012), Collegiality and careerism trump critical questions and bold new ideas: A student’s perspective and solution. Bioessays, 34: 448–450. doi:10.1002/bies.201200001 http://onlinelibrary.wiley.com/doi/10.1002/bies.201200001/epdf

Von historischem Interesse: Ein Faksimile von Otto Warburgs ‘Forschungsantrag’ bei der Notgemeinschaft der Deutschen Wissenschaft (1921). Das waren noch Zeiten! https://dirnagl.com/?s=warburg



Einsichten eines Wissenschaftsnarren (2): Zu Risiken und Nebenwirkungen fragen Sie Ihren Bibliothekar

Ein wunderbarer Vergleich zwischen der ineffizienten Weise, in der wir heute wissenschaftliche Resultate veröffentlichen, und was alles möglich wäre, wenn wir die Möglichkeiten es besser zu machen, nutzen würden (von Romain Brette) : A vision of the post-journal world. http://romainbrette.fr/a-vision-of-the-post-journal-world/

Gleich nochmal Romain Brette, mit einfachen Schritten wie in diese wunderbare Welt kommen: http://romainbrette.fr/my-new-year-resolution-to-help-move-science-to-the-post-journal-world/

Björn Brembs  gibt Tips was man tun kann, wenn man den Zugang zu Journalen verloren hat. Weil die Subskription gekündigt wurde, oder man auf der falschen Seite des Paywalls sitzt. Alles halb so wild…: http://bjoern.brembs.net/2016/12/so-your-institute-went-cold-turkey-on-publisher-x-what-now/

Björn Brembs rät, alle Journalsubskriptionen zu kündigen: http://bjoern.brembs.net/2016/05/why-havent-we-already-canceled-all-subscriptions/

Björn Brembs‘ kritische Analyse der gegenwärtigen OA Initiativen: http://bjoern.brembs.net/2017/03/please-address-these-concerns-before-we-can-accept-your-oa-proposal/

Björn Brembs warnt, dass der Königsweg des OA (‚Goldener OA‘) mehr Probleme verursachen könnte, als er löst: http://bjoern.brembs.net/2016/04/how-gold-open-access-may-make-things-worse/

Und jetzt auch das noch: Björn Brembs, Deutschland’s prominentester Open Science Aktivist kündigt frustiert seinen Rückzug an: http://bjoern.brembs.net/2017/02/open-science-too-much-talk-too-little-action/

SCI-HUB: legal – iIlegal – scheissegal. Über 40 Millionen Artikel zum Download.  Radikal illegaler Open Access.  https://dirnagl.com/?s=sci-hub

Oder nicht ganz so umfänglich, dafür legal, und sehr bequem mit Browser Extention: ‚Skip the paywall‘  Get full-text of research papers as you browse, using Unpaywall’s index of ten million legal, open-access articles.  http://unpaywall.org/

The Costs of Flipping our Dollars to Gold. Interview about the Pay It Forward report. The first comprehensive analysis of how a decisive push to Gold OA would impact the cost of publication for major North American research institutions. https://scholarlykitchen.sspnet.org/2016/08/24/the-costs-of-flipping-our-dollars-to-gold/

Elsevierleaks: Leaked Elsevier contract reveals pushback: http://scienceguide.nl/201703/leaked-elsevier-contract-reveals-pushback.aspx

Thomson-Reuters verkauft den Impact Factor an Venture Kapitalisten: http://www.marketwatch.com/story/thomson-reuters-announces-definitive-agreement-to-sell-its-intellectual-property-science-business-to-onex-and-baring-asia-for-355-billion-2016-07-11

PIPredictor – Predict your probability to become a Principal Investigator (PI) http://pipredictor.com

Berliner Erklärung zu Open Access. 60% bis 2020!: https://openaccess.mpg.de/Berliner-Erklaerung

Das BMBF zu OA: https://www.bmbf.de/de/open-access-das-urheberrecht-muss-der-wissenschaft-dienen-846.html

Missbrauch der Marktmacht. TAZ Artikel zum DEAL Konsortium und Elsevier: http://www.taz.de/!5370972/


Auch lesenswert:

Brembs BButton KMunafò M. Deep impact: unintended consequences of journal rank. Front Hum Neurosci. 2013 Jun 24;7:291. http://journal.frontiersin.org/article/10.3389/fnhum.2013.00291/full

Kriegeskorte N, Walther A, Deca D. An emerging consensus for open evaluation: 18 visions for the future of scientific publishing. Front Comput Neurosci. 2012 Nov 15;6:94. http://journal.frontiersin.org/article/10.3389/fncom.2012.00094/full

Khan R, Goodman L, Mittelman D. Dragging scientific publishing into the 21st century. Genome Biol. 2014 Dec 11;15(12):556. https://genomebiology.biomedcentral.com/articles/10.1186/s13059-014-0556-2



Einsichten eines Wissenschaftsnarren (1): Wie originell sind eigentlich Ihre wissenschaftlichen Hypothesen?

Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 2016 May 26;533(7604):452-4. doi: 10.1038/533452a

Button KS, Ioannidis JP, Mokrysz C, Nosek BA, Flint J, Robinson ES, Munafò MR. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013 May;14(5):365-76. doi: 10.1038/nrn3475

Button KS. Statistical Rigor and the Perils of Chance. eNeuro. 2016 Jul 14;3(4). pii: ENEURO.0030-16.2016. doi: 10.1523/ENEURO.0030-16.2016.

Dirnagl U. Sind die meisten Forschungsergebnisse tatsächlich falsch? Laborjournal 2014 7/8:38-41

Goodman SN, Fanelli D, Ioannidis JP. What does research reproducibility mean? Sci Transl Med. 2016 Jun 1;8(341):341ps12. doi: 10.1126/scitranslmed.aaf5027

Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124.

Kimmelman J, Mogil JS, Dirnagl U. Distinguishing between exploratory and confirmatory preclinical research will improve translation. PLoS Biol. 2014 May 20;12(5):e1001863. doi: 10.1371/journal.pbio.1001863.

Mogil JS, Macleod MR. No publication without confirmation. Nature. 2017 Feb 22;542(7642):409-411. doi: 10.1038/542409a.

Open Science Collaboration. Estimating the reproducibility of psychological science. Science. 2015 Aug 28;349(6251):aac4716. doi: 10.1126/science.aac4716.

Rheinberger HJ Experiment, Differenz, Schrift. Zur Geschichte epistemischer Dinge. Basilisken-Presse, Marburg/Lahn 1992, ISBN 3-925347-20-8.


und hier geht’s zum Blog:cropped-buzz.jpg