When I use a word . . . Research integrity sleuths

When I use a word . . . Research integrity sleuths

When I use a word . . . Research integrity sleuths


Recent years have seen the emergence of research integrity sleuths, who devote time to detecting misconduct in academic research publications, a major source of which comes from paper mills, satisfying the “publish or perish” urge. Their methods include detection of plagiarism, spotting red flags that mark the possibility of fraud, such as doctored and reused images, tortured phrases, unlikely collaborations, suspicious looking email addresses, and citations that appear irrelevant, and the use of statistical tests to detect fabricated data and lack of expected randomness.

The “publish or perish” doctrine

Being interested in why people write, and having surveyed the musings of several well known writers, including Alphonse Daudet, Joan Didion, E M Forster, Neil M Gunn, Rayner Heppenstall, Stephen King, George Orwell, and Alfred Perlès, I have come to the conclusion, following the trail of the cacoethes scribendi, first laid by the Roman poet Juvenal, that the main reason for the itch to write is the need for self-comprehension. Or, as Stephen King put it, “I write to find out what I think.”1

Among others who have tackled the question of why they write, the Canadian writer Mordecai Richler repeated Orwell’s quartet of reasons (egoism, aesthetic enthusiasm, historical impulse, and political purpose) and recognised being “driven by some demon whom one can neither resist nor understand.” However, he might have seemed to have failed to realise the source of this as a need to know what he thinks, had it not been for a throwaway remark in which he blessed the days on which “drawing on resources unsuspected, you pluck ideas and prose out of your skull that you never dreamt yourself capable of.” In other words, almost without realising it, finding out what he thinks.2

Richler, however, had one advantage that many other novelists have not had, having taken what he called a “reverse sabbatical,” namely working in academia for a year, which gave him an insight denied to others: “If stand-up comics now employ batteries of gag writers because national TV exposure means they can only use their material once, then professors, playing to a new house every season, can peddle the same oneliners year after year, improving only on timing and delivery. For promos, they publish. Bringing out journals necessary to no known audience, but essential to their advancement.” In short, without naming it, he had identified the “publish or perish” doctrine.

Following the trail further, I started to explore this phenomenon, which many dislike but appear to have become stoical about,3 defining it and describing its origins and adverse effects,45 and also describing proposed solutions to the consequent problems to which it gave rise.6

“Publish or perish”—problems and potential solutions

The problems that arise from the doctrine include, as I have previously described5: the increasing load of papers that journal editors have to deal with; increasing difficulty in finding reviewers; deterioration in the quality of papers being submitted; an increase in research misconduct, reflecting an increasing frequency of violations of academic integrity, i.e. matters of academic dishonesty or misdemeanours, such as fabrication or falsification of data, plagiarism, inappropriate gift, guest, and ghost authorship, misuse of statistics, concealing the use of AI, withholding information, for example about conflicts of interest, and selective or inaccurate citation; a burgeoning number of paper mills and predatory journals; wasted resources; a reproducibility crisis; stifling of innovation; increasing numbers of retractions of published work; increased burnout among senior academics and an increased reluctance on the part of trainees to enter research because of a fear of the pressures involved; and diversion of attention from teaching to research.

Most of the published suggested solutions to all this are no more than vague platitudes, which I have discussed in detail,6 and which I believe are unlikely to be of any use whatsoever in stemming the ever swelling tide of research misconduct of all kinds. This is primarily because the proposed solutions are likely to be ignored by those who are determined to indulge in acts of misconduct—in other words, the miscreants, the progenitors of many of the problems.

The only potentially effective solution that I have identified is eradication of the doctrine itself. In other words, dissociating academic publication from the rewards that it brings in the way of status and advancement. A good start could be made by abandoning modern forms of research assessment for the purposes of academic funding. But this is a solution that I do not expect to see occur, desirable though it may be.

Detecting academic misconduct

If little or nothing can be done to deter academics from misconduct, the next best thing is to detect and punish it when it occurs. In pursuit of this, it has been suggested that substantial violations of research integrity should be independently investigated by appropriately resourced specialists, completed rapidly, and followed by corrective action or exoneration as appropriate.7 The authors of this advice, all well known in the field of academic integrity, provide a clear definition of what they mean by “substantial,” define “rapidly” as being within a year, and describe other aspects of their suggestion in detail.

What methods are available?

Plagiarism

Plagiarism is often a hallmark of the outputs of paper mills, and different types of software are available to detect it, although several of them come from the same few companies, albeit marketed under different names. However, doubts have been cast on the ability of AI tools to detect plagiarism reliably,8 which concurs with my own limited experience. Although one would expect the available AI tools to improve, this is a double edged sword—AI can render plagiarised texts undetectable by rewriting them to appear to be original.9

A possibly better way of detecting some forms of plagiarism is to look for what are called “tortured phrases,” i.e. phrases that have been translated into congruent phrases in the same language but which themselves mean nothing.10 I don’t know how easy they are to spot, although I do know that they are not always easy to translate back to their originals. Try translating these examples before looking at the answers below:

• “bosom malignancy”

• “discourse acknowledgement”

• “enormous information”

• “individual computerised collaborator”

• “invulnerable framework”

• “motor vitality”

• “versatile organisation”

• “counterfeit consciousness”

• “man-made brainpower”

[Answers: breast cancer; voice recognition; big data; personal digital assistant; immune system; kinetic energy; mobile network; artificial intelligence; artificial intelligence. How did you do?]

My own suggestion of a tortured phrase for “artificial intelligence” is “fake news,”11 although I haven’t seen it used anywhere, at least not yet.

So, watch out for tortured phrases. Guillaume Cabanac’s Tortured Phrases Detector (https://dbrech.irit.fr/pls/apex/f?p=9999:24) will help.

Fake pictures

Certain forms of images are susceptible to fakery, such as Western blots. Experts at spotting cases of faked images have, for example, led to the discovery that many such exist in the field of Alzheimer’s disease, throwing doubt on the amyloid hypothesis of its causation.12 All this is extensively documented in Charles Piller’s recent book Doctored.13

Red flags

Since at least the 16th century, ships have been in the habit of hoisting a red flag to indicate their readiness for battle or at least as a symbol of defiance, and the habit became adopted by others involved in combat. By the middle of the 18th century the red flag therefore became generally associated with danger and acted as a warning of it or as a signal to stop.14 In modern metaphorical use it can mean anything that acts as a signal that something is not right.

Since the products of paper mills often contain several features that mark them out as fraudulent, one can look for such features—red flags. These include unlikely collaborations,15 suspicious looking email addresses that don’t match the authors, and citations that appear irrelevant, particularly those that refer to other publications from paper mills.16

Red flags such as these have been used to estimate the frequency of publications emanating from paper mills and the countries that are most commonly involved, reportedly Russia, Turkey, China, Egypt, and India.17

The STM Integrity Hub (https://stm-assoc.org/who-we-are), initiated by the International Association of Scientific, Technical and Medical Publishers in the Netherlands in 2023,9 combines many different methods in its search for features that mark products of paper mills and duplicate publications. In the USA the Office of Research Integrity (https://ori.hhs.gov) undertakes similar activities.

Statistical tests

From time to time, suspicion may be cast on published data through statistical testing. Famous cases include Ronald Fisher’s suggestion, since then much discussed and even disputed,18 that Gregor Mendel’s results on the inheritance of characteristics by garden peas were too good to have been legitimately obtained, even though they were correct. The complicated case of Cyril Burt’s results on the inheritance of intelligence, based on supposed studies of twins, has also partly rested on statistical interpretations, but only in part, and it remains controversial.19 More recently, doubt has been cast on some of the work published by the late Hans Eysenck in his collaboration with Ronald Grossarth-Maticek, in part based on suggestions of statistical implausibilities.20

Benford’s law, which relates to the way in which digits are distributed in data, and Carlisle’s method, which tests for divergence from randomness of data in clinical trials, are methods that have been used to test the plausibility of datasets.21

However, these are difficult areas, hard for the non-statistician to understand and often susceptible to varied interpretation.

A final comment

The word “sleuth” comes from an old Norwegian word slōð, meaning a track or a trail. A sleuth-hound was therefore a dog, a type of bloodhound, that followed a trail. In the late 19th century this was shortened to “sleuth,” meaning a detective. It seems to have been first used in the title of a novella titled “Old Sleuth, the Detective; or, the Bay Ridge Mystery,” written by “Old Sleuth” (Harlan Page Halsey), which first appeared in 1872.22

In recent years, the term “sleuth” has been increasingly used to describe those who have ferreted out evidence of academic misconduct. The earliest example that I have found dates from 2022,23 although the sleuths’ activities have antedated the use of the word to describe them.

The incidence of academic fraud has previously been thought to have been relatively low, but its extent has been underestimated1724 and it is on the increase. Using whatever expertise we have, we should all follow the example of Ro,25 and turn ourselves into research integrity sleuths.

References

  1. Cabanac G, Labbé C, Magazinov A. Tortured phrases: a dubious writing style emerging in science. Evidence of critical issues affecting established journals. arXiv 2021 Jul 12: 2107.06751v1.

  2. Piller C. Doctored. Fraud, Arrogance, and Tragedy in the Quest to Cure Alzheimer’s. Icon Books Ltd, 2205.

  3. “red flag, n.” Oxford English Dictionary. Oxford University Press, September 2024, doi:10.1093/OED/4848481432.

  4. Sabel BA, Knaack E, Gigerenzer G, Bilc M. Fake publications in biomedical science: red-flagging method indicates mass production. medRxiv 18 October 2023. doi:10.1101/2023.05.06.23289563.

  5. Hoppenstand G. The Dime Novel Detective. Chapter 3: Old Sleuth Library 1885-1905. Old Sleuth. Bowling Green University Popular Press, 1982: 136-81.





Source link

rovorobs@gmail.com

Leave a Reply

Your email address will not be published. Required fields are makes.

Top