Triangulation! The Egyptians used it to build their pyramids. The Greeks developed a branch of mathematics out of it. Until the 19th century whole countries were charted in this way. Far into the 20th century ships have determined their position with it. To determine your position by triangulation you only need a set square and a protractor, which the surveyors call a theodolite, as well as the coordinates of two visible landmarks. It’s that simple!

Could it be that triangulation is also an important methodological approach in biology? A cure even for the replication crisis? Munafo and Smith recently postulated this in a commentary in Nature. Sociologists call it triangulation when they use two or more different methods to investigate one particular research question. If the results converge at one point, i.e. lead to the same result, this increases validity and credibility. Don’t we do this routinely in the experimental life sciences? Does the knock-out mouse have the same phenotype as one in which the signalling pathway was pharmacologically blocked? Do transcript and protein expression correlate with the phenotype?

Thus, basic biomedical research is familiar with ‘targeting’ a goal with different methods grounded in already established knowledge (the landmarks of the surveyor!). Are the results converging? Bingo, we have located the biological mechanism! Therefore it leaves many of us cold, if spoilsports with gradschool statistics argue that most studies in biomedicine must be false positive despite significant p-value. Because we don’t just rely on ONE result. Instead we triangulate by means of different approaches! In order to validate results, this might even be superior to replication. If something is simply repeated, it is not unlikely that a systematic error will be repeated too. This would make the result reproducible, but still not correct.

Were the skeptics wrong when calling out a crisis in biomedical research? Are we already doing the right thing?

Unfortunately, despite lots of triangulation in many laboratories, things are not so easy. As any geodesist will confirm, the principle of triangulation is simple, but exact determination of location by triangulation is no child’s play. Biomedical basic research often does not adhere to the rules of triangulation. First of all, we have to be sure that our ‘landmarks’ are biologically sound, not the result of false positive results, overinterpretation of results, or an artefact of experimental conditions. The geodesist may find this easier, as he can refer to verfied positions of the reference landmarks with high precision on a map. If we in biomedicine apply the protractor, i.e. if we use different methods in our experiment, we have to read off with high accuracy. This implies blinding, no flexibility in the selection of the data points to be used, etc. If the bearing results in an angle that does not fit into the concept (i.e. confirm our hypothesis), we must not simply ignore its value and move the theodolite a little to repeat the measurement. According to the motto: Let’s try another antibody! Or another pharmacological blocker! But if we must adapt our designs, which may happen in exploration, we need to justify our modifications and report them in the publication. In addition, the read angle must be of high precision. Unfortunately, this is usually not possible with low sample sizes because biological variance often is quite large. And the measured angle must actually exist, i.e. it must not be a false positive finding due to a low number of samples or an improbable hypothesis.

I submit to you that if surveyors were to ‘triangulate’ land as we do our experiments, they would produce maps that are very plausible to draw. They could also be printed, and they might even look pretty. But if a hiker were to follow them, he’d get lost.

However, when used correctly, triangulation can actually be the key to more robust results. That implies experiments with sufficient sample sizes, bias prevention by blinding, randomization, etc., predetermined inclusion/exclusion criteria, and publication of the results independent of the results. Then triangulation is very effective: the cumulative sample sizess of the experimental series of different methodological approaches can actually be lower than those of a single series with only one approach. And this even with the same or higher statistical power and higher external validity. This is difficult to quantify, because statistical power of the combined results of triangulation is hard to calculate. And external validity, i.e. the generalizability and representativity of results, is not really quantifiable.

Now that a biological phenomenon has been provisionally located by triangulation, what should happen next? Of course, you will want to make it known to the world in a publication. However, one should be aware of the still existing limitations of the findings obtained in this way. This should already make itself felt in the title, in which the study should be marked as explorative. And in the conclusions one should restrain oneself. A reference to now possible therapies in humans, or necessary revisions of text books is rarely appropriate. Only a confirmation by replication in other laboratories can create solid evidence for the existence and the true extent of an effect. This usually requires larger sample sizes than in the original experiment, and the study must be pre-registered. It is quite clear that this is only feasible, meaningful and practicable with a small number of findings. However, when it comes to deciding, for example, whether to move from animal experiments to human studies, this should be a matter of course. The Federal Ministry of Education and Research (BMBF) has recently published a call for preclinical confirmatory studies. This is a revolutionary initiative that will hopefully set a precedent: With other funders, but also with us scientists.

A German version of this post has been published as part of my monthly column in the Laborjournal: http://www.laborjournal-archiv.de/epaper/LJ_19_03/22/index.html