In the history of humanity, science has always been the driving force behind civilization and innovation. However, it always requires the investment of resources, with time being the primary factor. In the modern world of technological progress, science is expected to solve problems quickly and effectively without room for failure, particularly in the development of Artificial Intelligence (AI) technologies. The industry’s pressure on research for the rapid development of groundbreaking solutions is particularly high. The Slow Science approach suggests rethinking the current influential “publish or perish” mindset.
What is Slow Science?
The concept of Slow Science is part of the broader “Slow Movement” movement, asserting that life in the modern world has accelerated excessively in all areas. Therefore, it advocates returning to a pace that allows for savoring events and prioritizing quality over quantity. Slow Science is based on the view that science should be a steady, methodical process. Scientists should not be expected to provide “quick solutions” to societal problems. Slow Science supports curiosity-driven scientific research and opposes performance goals, such as the quantity of publications in prestigious journals and conferences.
Performance goals create pressure on those aiming for success in academic careers. The aphorism “publish or perish” best describes the situation concerning the pressure to publish. Successful publications bring attention to researchers and their institutions, aiding further funding. Activities that do not lead to publications, such as working with students, may lead to falling out of competition and losing one’s position. This pressure is particularly strong at research universities and is cited as a cause of the replication crisis, where scientific articles are published that replicate existing work, leading to lower overall publication quality.
An example of an opposing development is the Slow Science Academy, founded in Germany in 2010, promoting the need for deceleration in scientific work. It emphasizes that scientists themselves should understand that science takes time. Their “Slow Science Manifesto” urges to “Have patience with us while we think.”
Why is rushed science undesirable?
The consequences of rushed science can directly impact our lives. The pressure on scientists to find a solution to the COVID-19 pandemic was enormous. The successful testing of several vaccines may seem to contradict the idea of Slow Science. However, the effects of “rushed science” were apparent. Six months after the pandemic spread rapidly worldwide, several reputable journals published a study claiming that malaria drugs also help with COVID-19. Shortly after, it was revealed that the experiments were not conducted properly: there were no double-blind tests, and results were reported only for a small number of patients (excluding those who died). The article was retracted, but for the scientific community, it served as a signal: even esteemed journals in science can fall victim to rushed, results-oriented research.
Publication pressure in Machine Learning
The field of Machine Learning and AI often experiences the “publish or perish” effect. According to Yoshua Bengio, a leading scientist in this field, the first factor to mention is the emergence and popularity of the scientific platform arXiv, an online preprint collection where articles can be published without a review process.
This graph shows the number of new entries on arXiv in each month since August 1991 (after 29.4 years). The total number of submissions as of December 22, 2020 is 1,813,740.
The popularity of arXiv has been growing exponentially since it was founded. The articles submitted to the database cover several fields, including computer science (especially Machine Learning). But what is the ratio of research disciplines in the total amount of articles submitted?
Statistics of the arXiv submission rate. The chart on the left shows the number of new submissions per year for different research disciplines. The chart on the right shows the same data as on the left, but with the submission rates in relation to the total number of submissions for each year.
You can clearly see the strong growth in computer science-related articles. As of 2019, the proportion of computer science (cs) articles is 27.8% of all submitted articles. This is a signal that the pressure to publish in this rapidly developing field of research may be too high.
The second reason is the number of Machine Learning conferences. The community has largely shifted to conference-based publications due to the longer time commitment required for journal publication. According to guide2research.com, there are 960 Machine Learning conferences worldwide. Researchers usually have about two months between submission deadlines for major conferences, where many scientists strive for publication. The popular conference NeurIPS has experienced exponential growth in the number of submissions.
The NeurIPS shows exponential growth in the number of submitted research projects in the years 2014 – 2019.
This situation not only reflects the pressure on researchers proposing an article but also places high demands on reviewers. Papers are passed from one place to another, hoping that the “random selection of reviewers” will be more acceptable in the next attempt. Most Machine Learning conferences show similar growth in the number of submissions as NeurIPS.
In his February 2020 post, Yoshua Bengio suggests reconsidering the organization of Machine Learning conferences. He recommends presenting fewer and much “slower” developing works and leaving the main publication process to journals. This field’s current state particularly affects young researchers who need to establish themselves and their research within the discipline. Overall, a doctoral student today must, in order to obtain a degree, publish at least 50% more publications than before the 1990s. In addition, the reproducibility of work in the field of Machine Learning is extremely low – anyone trying to obtain benchmarks for a newly developed solution approach is often unable to reproduce previously published results (see Quantifying Independently Reproducible Machine Learning and Artificial intelligence faces reproducibility crisis).
Should Slow Science find its way into Machine Learning research?
Overall, the pressure and volume of researchers in certain areas, for example in deep learning, is already leading to noticeable problems with regard to the research process and the quality of published articles. So, should the process follow the Slow Science motivation? Perhaps the opinion of a renowned scientist can answer this: physicist Sir Peter Higgs, namesake of the Higgs boson, was quoted in 2013 as saying that academic expectations since the 1990s had probably prevented him from both making his groundbreaking research contributions and gaining tenure. “It’s hard to imagine how I could ever have enough peace of mind in the current climate to do what I did in 1964,” he said. “I wouldn’t be able to get an academic job today. It’s as simple as that. I don’t think I would be considered productive enough”.
For more information on the topic:
Prof. Dr. Katharina Morik, Head of the Chair of Artificial Intelligence and the Collaborative Research Center 876 at TU Dortmund, and spokesperson for the Lamarr Institute Competence Center, addressed publication pressure in ML research in her lecture “Publish or Perish” as part of the international summer school “Resource-aware Machine Learning.”
By loading the video, you agree to YouTube’s privacy policy.
Learn more
