Pursuing the Next Big Thing: Research, replication, and review
Dr. Jo Roislien is a famous international science communicator who reaches a broad audience by appearing regularly on radio, television, and the printed press. He supplies lectures on the communication of sophisticated topics, skill dissemination, and his own research. He is a Norwegian mathematician, biostatistician, and researcher in medicine, and holds a PhD in geostatistics from the Department of Petroleum Engineering and Applied Geophysics at Norwegian University of Science and Technology (NTNU). Presently, he is an Associate Professor at the Department of Health Sciences, University of Stavanger, with extra posts in the Department of Biostatistics and the Center for Addiction Research at the University of Oslo. For his total bio in which titles of his productions are also listed, please see the very first part of his interview.
In the first part of a three-part interview, Roislien provided meaningful insight into the nature of data and statistical analysis. Here in his next segment, Roislien discusses his views on the challenges in journal publishing, basic versus applied research, and replication studies.
As an academic author, can you discuss your opinion about current challenges in research or journal publishing?
In the mathematical sciences we tend to have a relatively relaxed relationship with the influence factor. Because we all know it is a vague measure of the influence of the journal. In mathematical sciences, truth does not have an expiry date, so citations tend to cumulate leisurely, and for some papers they proceed seemingly forever. Some papers have massive amounts of citations, most have few or none.
The influence factor is an expiry date type measure. Fair enough. It won’t reflect every influence. But, honestly, did you indeed think it would? Finding one measure for “importance” is obviously unlikely. But it doesn’t automatically mean that the influence factor is worthless. In the same way as BMI tells you something about your health, but not everything. Your salary tells you something about your economy, but not everything.
The influence factor is sometimes viewed as a problem in itself, but the problem is when this elaborate, high-dimensional problem is being diminished to a one-dimensional measure.
I find it more annoying tho’ when people criticize without coming up with an alternative solution. I have an engineering background, and engineers concentrate on coming up with solutions. Obviously the world isn’t ideal. Of course it can be improved upon. We know. Now instead, can you suggest something better?
As an academic, peer review is part of my everyday life. I submit papers for review; I review papers other people have submitted. And I must admit that I am a bit astonished by the poor quality of remarkably many of the papers I am asked to review. People tend to think that just because you spent a lot of time on something that qualifies as research. And these poor papers I am asked to review never come from high influence factor journals, but from fresh journals, low influence factor journals, journals who brag about very quick referee process. Things like that. Bad science is bad for science. So some filter is necessary.
To me it’s evident that we are gravitating towards a world where all research papers are available to everybody. But it is a process. Even online journals cost money. Peer review processes must still be coordinated. Software updated. Editorial meetings. We must permit these things to take time. Not because we do not want switch, but because we must permit ourselves to think these things through decently.
You seem to promote learning by discovery. What are your views on basic research versus applied research?
When working on my doctorate, in geostatistics and stochastic modelling, my supervisor always encouraged me to think for myself. We would discuss some possible strategies and approaches for solving a particular problem, which would generally imply throwing me into deep mental waters without a life vest, to which I would often mumble “I’ll go have a look in the library,” and my supervisor would go “No-no-no! Attempt for yourself.” And so I did. The result was that coming up with a possible solution to the problem at arm was sometimes slow, but I learned a lot. No, wrong choice of word. I understood a lot. Big difference.
I tend to take the same treatment with my own students. And my kids. Even in the public domain. When we made the hugely successful math TV series “Siffer” (“Digits”) on Norway’s largest national broadcaster NRK1 (the series rated 40% of TV viewers and was nominated for two Norwegian Emmys) we took the same treatment. Rather than focusing on theory and hypothetical situations we went out into the real world exploring it through the eyes of mathematicians and statisticians. We explored mathematical graph theory by going to a high school and constructing a make-out-map: Who has made out with whom? And what does that tell us about the spread of infectious diseases? Things like that.
Some of my peers think that my treatment is populist, but I find that attitude overly passive. To me it is all about inviting people into the world of skill by siding with the natural way for humans to learn. Playing. Being nosey. Discovering. How does the world work?
You cannot learn football by watching football on TV. You cannot learn to play an instrument by listening to music. You cannot learn statistical analysis without actually analyzing. You have to do.
And this goes for both basic and applied research. I love basic research, and respect it a lot, and I honestly thought that that would be my career. My life has just brought me elsewhere.
In your field, how significant are replication studies and is there a need for more?
There is always a need for replication studies. One investigate is an anecdote. Give me two, ten, a hundred studies that all point in the same direction. Now we’re talking.
We tend to leave behind that the scientific theories that are the foundation of our society, Newton’s laws of movability, or Evolution, are not one person’s view, one single awesome investigate. They are piles and piles of studies providing the same result. Newton’s laws of movability are still developed from the ground up and tested and retested over and over again. Every engineering student in the world has deduced Newton’s law of universal gravity and performed experiments to quantify the gravitational pull of the earth. Evolution has been confirmed in literally millions of studies across the world, but that hasn’t stopped scientists from continuously gathering more and more evidence.
A lot of resources are being put into medical research now. And medical research results are all over the news. The problem (if that is the right word) with medical research being a relatively youthfull research field compared to STEM (where I have my background) is that there is so much fresh ground to cover, that in medicine ‘exploratory’ tends to be promoted much more than ‘confirmatory.’ So researchers in the field all pursue the Next Big Thing. The fresh result, the fresh hypothesis, the fresh treatment. This pursue for ‘new’ leaves us in some sort of a vacuum, where the frontier seems to be moving forward in a rapid rhythm, but not enough studies are being conducted to confirm, or refute, fresh findings, to establish whether the frontier is indeed moving forward at all, or in the right direction.
I wish I could live a hundred years from now, and look back on today. How will it look? Which results will stand? Which ones will not?
Read the next and final segment of this interview with Dr. Jo Roislien.