What's Wrong With Science? --- A series of high-profile retractions has raised questions about the process used by scientific and medical journals to decide which studies are worthy of publication.

"The latest in a series of high-profile retractions of research papers has people asking: What's wrong with science?

Scientific and medical journals use the peer-review process to decide which studies are worthy of publication. But a string of questionable or allegedly fabricated research has made it into print. The problems were exposed only when outside researchers scrutinized the work and performed a job that many believe is the responsibility of the journals: They checked the data.

The number of retractions has been rising for years. The website Retraction Watch cataloged more than 5,400 retractions for 2022, up from about 120 in 2002. 

On Tuesday, a paper claiming the discovery of a room-temperature superconductor was retracted by the journal Nature after physicists flagged problems with the study and Nature conducted its own investigation.

The journal's decision to retract the paper validated the barrage of criticism it had received almost immediately upon publication. But it didn't assuage critics of the peer-review process.

"Every serious scientist I know who looked at this paper thought it shouldn't have been published from the very beginning," said Peter Armitage, a Johns Hopkins University physicist. "I think what it says is that if we can't trust the journals, we do need to think about alternative systems."

Magdalena Skipper, editor in chief of Nature, contends that the role of peer review is misunderstood. Typically, reviewers are working scientists tapped by journal editors to critique submissions and recommend whether they should appear in print. Their reviews are almost always provided for free as a service to the scientific community. And to facilitate candid assessments, their identities are usually concealed.

Journal editors sift through submissions looking for studies that will significantly advance their field, then assign reviewers to evaluate the most promising ones for strength of conclusions and rigor of experimental methods. But journal editors acknowledge that errors or fraud can escape notice because reviewers don't audit underlying data sets.

That's not their job, according to Skipper. "I would not want to think of my peer reviewers on the papers as some kind of police squad catching mistakes," she said.

Researchers who take it upon themselves to parse peer-reviewed papers and expose problematic data believe that the journals are shirking a fundamental responsibility.

"One of the functions I think they think they have is to increase the chance of what we're reading is true and of high quality," said John Carlisle, an anesthesiologist and editor at the British journal Anaesthesia who has been flagging bad data in medical studies for nearly two decades. "I don't see how they can turn around and say, 'Well, it's not our job to actually check the data.'"

While only a small fraction of the millions of studies published every year are retracted, when questionable research does make it into the pages of a prestigious journal, the consequences can be severe and long-lasting.

The paper that famously -- and incorrectly -- linked autism to vaccines in 1998 wasn't retracted by The Lancet until 2010. In the interim, it sowed the seeds for decades of powerful vaccine distrust that experts said likely drove vaccine hesitancy even during the Covid-19 pandemic.

Scientists and journal editors say that several factors tax the system. More papers than ever are being published, overloading journal editors and reviewers. The most significant papers are often the most novel -- and therefore more difficult to review, because the material covers new and untested ground. And researchers under pressure to advance their careers are eager to supply the kind of notable findings that attract the attention of journals, tempting some researchers to cut corners or fabricate results.

"Having a Nature paper is the kind of thing that gets you tenure at most universities," said Melinda Baldwin, a historian of science at the University of Maryland, College Park, who studies publishing.

The Nature paper about the room temperature superconductor was the third major retraction in 14 months of studies involving physicist Ranga Dias at the University of Rochester, including another in Nature and one in Physical Review Letters. Neuroscientist and former Stanford University president Marc Tessier-Lavigne had three papers withdrawn this year, one by the journal Cell and two by the journal Science.

And Harvard Business School professor Francesca Gino had three papers retracted this year after an investigation by the university, including two published in Psychological Science and one in the Journal of Personality and Social Psychology. A fourth paper that Harvard cited for faulty data had already been pulled by the Proceedings of the National Academy of Sciences in 2021.

Each of these papers had passed peer review. And in each case, the retractions were spurred by scientists who, on their own, dug into the studies after publication and found that the underlying data was lacking. One suggestion for fixing peer review that comes up frequently is to pay reviewers. But journals are reluctant to add additional costs, said Holden Thorp, editor in chief of the Science family of journals: "It's very hard to think through the economics of how that's going to work."

Another is to bring some of the data-checking in-house, while continuing to rely on outside reviewers to assess the scientific design and arguments in new work. "Most journals should be checking more than they are now," said Simine Vazire, a psychology researcher at the University of Melbourne and the incoming editor in chief of Psychological Science. She plans to add scientists to her staff to check data and statistics in new submissions.

As an editor at Anaesthesia, Carlisle routinely audits patient data underlying clinical trial studies, looking for signs it was falsified. One in three papers has enough false data to make him disbelieve the results.

"If I was paid per hour, I'd be a very rich man," he said, adding that it should be the obligation of journals, not unpaid reviewers, to ensure the quality of published work. "Why are the publishing houses earning millions of dollars?" Carlisle said. "Why are people paying the journals unless the journals are doing the job?"" [1]

Two ideas of science coexist these days . One idea is a leftover, an appendix, from times before ours. Scientists  were noble, rich, smart, and rare human beings, pursuing knowledge for free, as a hobby or getting paid very little then. This is how we still do peer review today - for free. This is how you publish in Nature, and get a position at university for the rest of your life. You use fiction, fairy tales to enrich yourself. Sometimes it is enough just to fake data for that.

Our main reality is different now.  Science is a risky investment. Countries, universities and firms, when they are able to afford it, invest in scientific ventures, hoping that some of them will be wildly good. Scientists are professionals now, like many others. Any bad scientific production in the worst case kills people, like in the story with the vaccines "causing" autism, in the best case - leads to waste of money.

Therefore it is time to professionalize the audits of data underlying the results of the studies. For such work countries, universities and firms should split their money for research into two equal parts. The second part should be spent on the audits of data underlying the results of the published  studies. Scientists should compete for money needed to perform these audits. Databases of scientific results should have data attached, showing who audited the data underlying the results of the study. Countries, universities and firms doing a poor job of auditing the research that they financed should be known to the world, and the research based on their money should be considered cutting corners, and therefore too risky to use. Researchers performing data audits should be encouraged to publish their results in the same Nature where the first results were published, and automatically receive a permanent job with unlimited tenure at some universities. You have to be talented to reproduce important new research.

1. REVIEW --- What's Wrong With Peer Review? --- A series of high-profile retractions has raised questions about the process used by scientific and medical journals to decide which studies are worthy of publication. Subbaraman, Nidhi.  Wall Street Journal, Eastern edition; New York, N.Y.. 11 Nov 2023: C.5.