The opposite of extreme self-citing is self-plagiarism (either out of ignorance, to avoid extreme self-citing on ground-breaking research, or with malicious intent: passing the same paper to multiple journals as a new result).
> The rate of duplication in the rest of the biomedical literature has been estimated to be between 10% to 20% (Jefferson, 1998), though one review of the literature suggests the more conservative figure of approximately 10% (Steneck, 2000). https://ori.hhs.gov/plagiarism-13
If work by another author was enough to inspire you and add a reference, then your own previous work should certainly qualify, if it added inspiration to the current paper.
(Like PageRank, it is very possible to discount internal PR/links under external links, and when you also take into account the authority of the referencer, you avoid scientists accumulating references from non-peer reviewed Arxiv publications).
Self-citation is appropriate for a new paper that builds on the results of a previous paper. But in evaluating how influential a researcher is, it makes sense to exclude self-citation, while being careful to avoid any implication that self-citation is wrong.
The core problem here is that universities think that citation statistics are a useful metric to evaluate the quality of the work of a scientist. There's plenty of evidence that this is not the case or that even the reverse may be the case [1], but this idea refuses to die.
It sucks as a metric but it does have some rough correlation in most cases, and I'm not aware of any better easily measurable metric - if you have one in mind, it'd be great to hear. The alternative of having a bureaucrat "simply judge quality" IMHO is even worse, even less objective, and even more prone to being gamed.
The main problem is that there is an objective need (or desire?) by various stakeholders to have some kind of metric that they can use to roughly evaluate the quality or quantity of scientist's work, with the caveat people outside your field need to be able to use it. I.e. let's assume that we have a university or government official that for some valid reason (there are many of them) needs to be able to compare two mathematicians without spending excessive time on it. Let's assume that the official is honest, competent and in fact is a scientist him/herself and so can do the evaluation "in the way that scientists want" - but that official happens to be, say, a biologist or a linguist. What process should be used? How should that person distinguish insigtful, groundbreaking novel and important research from pseudoscience or salami-sliced paper that's not bringing anything new to the field? I can evaluate papers and people in my research subfield, but not far outside of it. Peer review for papers exists because we consider that people outside of the field are not qualified to directly tell whether that paper is good or bad.
The other problem, of course, is how do you compare between fields - what data allows you to see that (for example) your history department is doing top-notch research but your economics department is not respected in their field?
I'm not sure that a good measurement can exist, and despite all their deep flaws it seems that we actually can't do much better than the currently used bibliographic metrics and judgement by proxy of journal ratings.
Saying "metric X is bad" doesn't mean "metric X shouldn't get used" unless a better solution is available.
I think a problem here is Goodhart's law: "When a measure becomes a target, it ceases to be a good measure." [1] And it seems like there's an element of the streetlight effect [2], too; sometimes a bad metric really is worse than no metric.
Also, I really question your notion that people outside a field should be able to evaluate the quality of someone's work, especially in academia, where the whole point is to be well ahead of what most people can understand. That theory seems like part of managerialism [3], which I'll grant is the dominant paradigm in the western corporate world.
I understand why a managerialist class would like to set themselves up as the well-paid judges of everybody else. But I'm not seeing why anybody would willingly submit themselves to that. It's a commonplace here on HN that we avoid letting managers make technical decisions, however fancy their MBA, because they're fundamentally not competent to do it. That seems much more important for people doing cutting-edge research.
> the whole point is to be well ahead of what most people can understand
That’s not the case at all. Being at the leading edge of research should mean that you are creating new knowledge. That doesn’t imply that people cannot understand it. This expectation that laypeople cannot possibly understand science is one of the reasons so many papers are written so densely and obtusely. “They” can’t understand it anyway, right?
Feynman said if he couldn’t explain it to freshmen he didn’t understand it himself.
Why not simply use replication as a measure? Have your studies been replicated? How many other studies have you replicated?
Would both help solve the replication crisis, and resolve this problem.
Of course then you might have 10 000 studies replicating the same easy to do study... which is why the "score" should be reduced based on how many other times that study has been replicated.
The issue is how many researchers want to spend their time replicating the research of other people rather than doing their own original work. Getting funding is already incredibly hard, plus no-one is going to give you tenure or promote you for replicating the work of others.
Yes, bizarrely, reputation / trust is still the primary foundation of academia from a pragmatic perspective, even though it is the antithesis of science. At least some disciplines can have replication studies cross-culturally. It’s a hard problem to solve; knowledge is inherently a social/shared experience.
Trust is a curious one, it's not the antithesis of science, indeed it's required for science to actually work rather than simply be an idea.
You trust in people, in consistency of physical laws, in coherency of your own mind, in constancy of temporal flow, in so many things because - as the Pyrrhonist refrains - nothing is certain, not even this.
My understanding is that eigenfactor rates journals, not individual papers, so if somehow you get low-quality (whatever you want that to mean) papers into nature it has no independent way to realize that your specific paper is low quality. Also eigenfactor is biased towards favoring larger journals, which is not obviously a good thing. It would honestly be really cool if someone did page rank for individual papers. It seems like a much saner metric than anything that is currently used.
Isn't that how is done in academic circles? Maybe not quantatively but qualitatively surely tenure boards or hiring boards or student applicants notice such things.
"In 2017, a study showed that scientists in Italy began citing themselves more heavily after a controversial 2010 policy was introduced that required academics to meet productivity thresholds to be eligible for promotion"
Heisenberg's principle says that when you measure one thing, you lose track of others. I guess it's kind of complementary to this (except that it applies to very different things, obviously).
Publication standards are different between disciplines. Traditionally, a materials chemist publishes many times more papers than a structural biologist. With a metric like that, you run whole sub-disciplines out of town.
If it means that some academics are choosing to cite their own papers rather than keeping up to date on the literature of their field, there's likely to be some potential insights missed.
As someone who now works in astronomy, I'm not at all surprised at the high self-citation rate for the field. It is true that a lot of papers are published by large consortiums. For example, at LSST (where I work), if you have been working on the project for 2 years, you are considered a "builder" and added as an author to all major project wide papers.
Those papers, which tend to be long and full of great stuff, are cited a lot, and have hundreds of authors.
I wonder how many of these papers are where the first author has cited other papers where they are the first author. (Or really, at least the first few authors) It seems like for the data shown, it is just if anyone in the author list is anywhere in the author list of the citation?
Also for some research niches, you may be one of the few people writing papers on a subject. There's no one else to cite.
I do think there's some very valid points about bringing the person up to speed on previous research that brought them to the current paper. But I don't think those citations should really count as a citation in terms of metrics for how successful a scientist is.
To be honest, I find all the metric gaming about number of papers and citations to be ridiculous. I don't hear many people saying they want to write the best paper in their field, or something new. It all seems to be a numbers game these days. Academic career growth hacking, if you will.
> I wonder how many of these papers are where the first author has cited other papers where they are the first author.
Also, it's worth mentioning that countries which support PhD via publication essentially require you to conduct self-citing research. This is to show you've had a common thread between your research, and that the PhD can be defended as to have all the papers be considered to be on the same subject.
I'm not Italian... and am not meeting any productivity threshold.
But my work is incremental, and I obviously don't want to repeat what I said in a different paper, so I cite earlier work in later work. TBH, I don't think it's possible to avoid self-citation unless:
1. Your research is so popular that by the time you need to cite it, it's been surveyed, or improved upon, or otherwise adapted. 2. You switch research subjects relatively often. 3. You publish "blocks" of work, each based on fundamentals in your field established by others - and they're not incremental.
Perhaps it would be useful for reviewers to point out which citations do not contribute to the paper? It really is a tough problem. If someone is toiling along in some niche they have carved out, they and their colleagues may be the only one working in that space. That leads to a lot of cross citation and self citation.
That said, if you publish paper A, and then cite it in paper B which builds on that work, then in paper C you really only need to cite paper B if you're building on the work, not B and A. It might make for in interesting data set to plot out those sorts of relationships.
As a reader I personally prefer if they do a more complete set of citations, instead of making me follow up a multi-step chain to dig them up, as if I'm a compiler resolving transitive dependencies. I like little history-map sentences like: "This technique was introduced by Foo (1988) and recast in the modern computational formalism by Bar (2009); the present work uses an optimized variant (Bar 2012)."
You could just cite the last paper here, which is the only one used directly, and which presumably itself cites the earlier papers. But it's more useful to me if you include the version of the sentence that cites all three and briefly explains their relationship.
> if you publish paper A, and then cite it in paper B which builds on that work, then in paper C you really only need to cite paper B if you're building on the work, not B and A.
Logically I agree with you, but a lot of academics seem to believe differently when it comes to citing other people's work, and if we are to go by that logic (which a lot of people are inevitably forced to do), I don't see why one should treat their own work any differently.
I found this situation regularly when going down the rabbithole of the anti-vaxx or anti-5g people. One "scientist" makes a highly dubious claim, thousands of nutjobs cite this one scientist, "scientist" then goes on to cite articles that cites their work. I'm basically waiting to find Alex Jones cited in a serious article at this point.
In which case you have to ask yourself, are you so brilliant that you’ve found an important topic that no one has considered yet, or have all the brilliant people already figured out that topic isn’t worthy of study?
It’s the same with the startup world. If you’re the only one doing a thing, are you brilliant or foolish?
Most scientists work on topics that are quite niche. Most of those topics lead to nothing. A lot of good research took years to ripen enough to be of actual value. A lot of popular topics started out in a niche. Most of mathematics took dozens of years to fully come to fruitition. Can you decide beforehand which one will be the next big thing?
Today, most scientists go for the popular topics and whatever is on the government research plan to get funding.* Whenever the wind changes direction they change their topics because they need that funding.
*: This might seem to contradict with the statement that most scientists work on nice topics. But only on the surface. In order to get published you have to do something novel. So, you choose a popular topic and then research a rather unpopular side aspect on it like how a specific chemical behaves when applied to the popular topic. If you're successful you publish and continue. Citations come later or they don't but the next round of funding comes with publishing. After a few years without many citations you move on to the next thing.
On government plans often you need to publish and then it's done. The citations only matter long-term if at all. Most scientists don't achieve anything of greater value. They are happy if they can publish at all. If the institute has a few scientists with a high citation count it carries all the rest of them.
No, the answer is that there is a limited number of scientists and a limitless number of research directions. This doesn't have to be correlated with brilliance. In fact, it can be easier to research some of the less popular paths because there is less competition and more low-hanging fruits.
It seems like PhD candidates work on peripheral elements of their sponsor/tutor/professor's work, of that professor at some point is going to make a significant step then one of those PhDs will be along for the ride; not necessarily the genius one.
Two economists are walking down the street. One sees a $20 bill and starts to bend over to pick it up. The other economist says "Don't bother - if it were really worth $20, someone else would have picked it up."
There's countless numbers of scientists that made great strides working on topics others deemed rediculous. Heck, many of the Nobel prize winners were ridiculed by their colleagues as borderline wack-jobs at the time they were working on their research. Even after winning the prize, some still were with their later work (Crick's search for consciousness comes to mind, and why it would be so worthless a search does not).
If anything, the hubris of the scientific community would be as deafening as the pseudo-science BS and hold back progress just as much if not more except for one key thing: the scientific method.
Luckily, we have a process by which crackpots get differentiated from geniuses. So let's not leave $20 on the ground assuming others would have picked it up, especially when that $20 represents collective progress for the entire species.
> Heck, many of the Nobel prize winners were ridiculed by their colleagues as borderline wack-jobs at the time they were working on their research. Even after winning the prize, some still were with their later work (Crick's search for consciousness comes to mind, and why it would be so worthless a search does not).
Do you have any good examples of being considered wack-jobs before their winning?
Semmelweis. He never received the Nobel prize, but I think he counts towards the point.
> Dr. Ignaz Semmelweis discovered in 1847 that hand-washing with a solution of chlorinated lime reduced the incidence of fatal childbed fever tenfold in maternity institutions. However, the reaction of his contemporaries was not positive; his subsequent mental disintegration led to him being confined to an insane asylum, where he died in 1865.
It isn't too hard to find examples of scientists who were ridiculed for their ideas and eventually win the Noble prize:
>...Stanley B. Prusiner, a maverick American scientist who endured derision from his peers for two decades as he tried to prove that bizarre infectious proteins could cause brain diseases like “mad cow disease” in people and animals, has been awarded the ultimate in scientific vindication: the Nobel Prize in medicine or physiology.
>...Prusiner said the only time he was hurt by the decades of skepticism “was when it became personal.” After publication of an especially ridiculing article in Discover magazine 10 years ago, for example - which Prusiner Monday called the “crown jewel” of all the derogatory articles ever written about him - he stopped talking to the press. The self-imposed media exile became increasingly frustrating to science journalists over the past decade as his theories gained scientific credibility.
>....The recent 2011 Nobel Prize in Chemistry, Daniel Schechtman, experienced a situation even more vexing. When in 1982, thirty years ago, he made his discovery of quasicrystals, the research institution that hosted him fired him because he « threw discredit on the University with his false science ».
>...He was the subject of fierce resistance from one of the greatest scientists of the 20th century, Linius Pauling, Nobel Laureate in Chemistry and Peace Nobel Laureate. In 1985, he wrote: Daniel Schechtman tells non-sence. There are no quasi-crystals, there are only quasi-scientists!
An example that is pretty well known is Barry Marshall
>...In 1984, 33-year-old Barry Marshall, frustrated by responses to his work, ingested Helicobacter pylori, and soon developed stomach pain, nausea, and vomiting -- all signs of the gastritis he had intended to induce.
>...Marshall wrote in his Nobel Prize autobiography, "I was met with constant criticism that my conclusions were premature and not well supported. When the work was presented, my results were disputed and disbelieved, not on the basis of science but because they simply could not be true."
It was Max Plank who said "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." - so this isn't a new issue and things are probably better now than they were in the past.
>In which case you have to ask yourself, are you so brilliant that you’ve found an important topic that no one has considered yet, or have all the brilliant people already figured out that topic isn’t worthy of study?
Imagine how many inventions we would have missed if all inventors had shared your mindset.
What? These are perfectly valid questions to be asking and do not inherently stop you from researching what you're working on.
I know some scientists that define their research direction by asking these questions first before pursuing an idea. Many great inventions like optogenetics or expansion microscopy came from this investigative strategy. It can help keep your resources and energy in check.
The purpose of PHDs are to move human knowledge forward. You have to do an analysis of something that, in all likelihood, nobody has done before (or not enough to be considered settled).
This is actually a very interesting question that might take one down the rabbit hole.
Acquiring knowledge (I should say 'beliefs about valid knowledge') and brainstorming (and certainly collaboration and getting an advisor to adopt you) appear to be social activities, as much as purely logical and analytic activities.
Social activities like this, for social or herd creatures, are subject to flock or swarming patterns.
Maybe all the brilliant people are swarming around a locus of interest? It's certainly a good way to have the population explore the ins and outs of a, well, locus of interest. It's also a good way to have a loner get shunned by wandering off and poking at an uninteresting pile of dung.
I guess my point is: why not both? (Mathematically, statistically, egotistically, I know the idea that I am the foolish one is almost certainly more likely to be the case)
You're seriously underestimating the number of things there are to study and overestimating how many people there are to do the study.
This is nothing like tech startups, where you have tons of people sharing a relatively small problem space (creating tech tools companies want).
Consider for a moment: there are well over 350,000 different species of beetles. Theres just too much to study and too few people doing the work to expect there to always be a plethora of external research to draw upon.
At the time, chytrids were about as obscure as a topic in science can be. Though fungi compose an entire organismal kingdom, on a level with plants or animals, mycology was and largely still is an esoteric field. Plant biologists are practically primetime television stars compared to mycologists. Only a handful of people had even heard of chytrids, and fewer still studied them. There was no inkling back then of the great significance they would later hold.
Longcore happened to know about chytrids because her mentor at the University of Michigan, the great mycologist Fred Sparrow, had studied them. Much yet remained to be learned—just in the course of her doctoral studies, Longcore identified three new species and a new genus—and to someone with a voracious interest in nature, chytrids were appealing. Their evolutionary origins date back 600 million years; though predominantly aquatic, they can be found in just about every moisture-rich environment; their spores propel themselves through water with flagella closely resembling the tails of sperm. Never mind that studying chytrids was, to use Joyce’s own word, “useless,” at least by the usual standards of utility. Chytrids were interesting.
The university gave Joyce an office and a microscope. She went to work: collecting chytrids from ponds and bogs and soils, teaching herself to grow them in cultures, describing them in painstaking detail, mapping their evolutionary trees. She published regularly in mycological journals, adding crumbs to the vast storehouse of human knowledge.
And so it might have continued but for a strange happening at the National Zoo in Washington, D.C., where poison blue dart frogs started dying for no evident reason. The zoo’s pathologists, Don Nichol and Allan Pessier, were baffled. They also happened to notice something odd growing on the dead frogs. A fungus, they suspected, probably aquatic in origin, though not one they recognized. An internet search turned up Longcore as someone who might have some ideas. They sent her a sample which she promptly cultured and characterized as a new genus and species of chytrid: Batrachochytrium dendrobatidis, she named it, or Bd for short.
That particular chytrid would prove to cause a disease more devastating than, as best as scientists can tell, any other in the story of life on Earth. After Longcore’s initial characterization, she and Nichol and Pessier proceeded to show that frogs exposed to Bd died. Other scientists soon linked Bd and its disease, dubbed chytridiomycosis, to massive, inexplicable die-offs of amphibians in Costa Rica, Australia, and the western United States. No disease had ever been known to cause a species to go extinct; as of this writing, chytridiomycosis has driven dozens to extinction, threatens hundreds more, and has been found in more than 500 species.
Almost overnight Longcore went from obscurity to the scientific center of an amphibian apocalypse. “Had I not been studying the ‘useless’ chytrids,” she says, “we wouldn’t have known how to deal with them.” Her research has been crucial—not only the initial characterization, but also her understanding of the systematics and classification of chytrids, which helped provide a conceptual scaffold for questions about Bd: Where did it come from? What made it so strange and so terrible? Why does it affect some species differently than others?
You need a source of trust in these systems. Journals used to have that role. They had high standards that were upheld by editors selecting only worthy publications. Today it seems that many journals aren't as trustworthy as they seemed to be in the past. It's also easier to spam the journals with your publication and to bullshit your way into publication. The incentives to publish a lot are also way higher now that your grant money is highly dependent on your citation count. Journals can publish more and easier and lower the standards for submission to earn more money. The system is basically eating itself and we haven't found a cure yet.
Filtering for self-citations is useful to identify the bubbles. But it is not sufficient to determine if those bubbles only contain hot air or if these scientists are actually working on something with substance in a narrow field where few others publish.
Interestingly enough, the published work [1] has two self-citations. So maybe there's something to that phrase, “the next work cannot be carried on without referring to previous work.”
My undergrad college physics professor (Fay Ajzenberg-Selove) introduced this metric back in the 50’s. She faced major sexism and bullshit claims against her productivity, and had to use this metric to prove her detractors wrong, that she was as good as or better than most of her male colleagues in terms of performing useful and interesting research, to earn herself a faculty position.
On a semi-related note, I occasionally look at the news items Google news suggests for me, and these include a significant portion of climate change denialists propaganda, including one shocked, shocked by Nature "suppressing academic freedom with this list" (and my searches are never for climate denialism).
Which is to say, these may be a few sciences but it seems like they significant resources behind them, somehow.
Strange, I use google news on a semi regular basis, and have the opposite experience; most are fairly bland coverage of climate science research announcements with the occasional hyperbolic doomsday stuff.
Regrettable that the article begins by outing a prof from an unheard of university in India, who probably publishes in low repute journals and conference. Ideally the citing malfeasance score should be weighted based on journal and conference reputation.
A particular interest of mine is database systems (and by extension, distributed systems) and I’ve noticed this pattern a fair amount when reading database-related CS papers. It tends to feel like a small circle of researchers citing each other. Don’t get me wrong, that doesn’t say much about the actual quality of said papers, but it’s a pattern that I’ve definitely noticed. As a result of this, I tend to also make sure that I pay attention to seemingly interesting papers with low citations; sometimes, it immediately makes sense why they have low citations, while, other times, I find myself fascinated by the research despite the low citations.
People who are working out their ideas far outside the mainstream have no one to cite but them selves. Some are quacks, to be sure, but sometimes a field is just not ready for their work because the utility of the idea is not easily apparent, or because it's perceived to be too risky. A field needs a healthy mix of the the curmudgeonly stubborn thinkers going their way no matter the cost, and those making steady progress on solvable problems.
Sure, but there is no 'pruning of the tree' in case a dead end is reached, so the citations stay allowing the quack to pretend they have more credibility than they do. In fact, the whole idea of these citations is to build credibility where there is none.
Right. The purpose of citation metrics is to measure the intellectual influence of a person's ideas. If an academic's works are never being cited by anyone else, the correct conclusion would be that their ideas have little influence -- regardless of how many self-citing papers they've published.
The need to step outside of mainstream views has been important for intellectual progress over the years, indeed. The problem we have now is we're stuck with a series of bad faith, polarizing and disingenuous views that "suck the oxygen out of" actual wide-ranging thinking - IE, climate denialists can still get a lot of money, anti-vaxers can pull in a lot of money from scams, etc.
Possibly Barry Marshall and Robin Warren who discovered H Pylori causes peptic ulcers and subsequently won a Nobel Prize. They were ridiculed by doctors and scientists for their theory.
Between the two authors here (McKinsey and Tarski), they cite each other 7 times, with 14 citations, giving 50% outgoing self-citation, however this paper also received incoming citations contemporaneously...
Not defending people who self-cite but here is an alternative explanation. There are many areas of science that have very few researchers working on the same or related problems. In addition papers tend to build on prior works of the same or related researchers. Over time we may see clusters of what look like "self-cited" papers. This is not abnormal.
Yup, I self-cite quite a bit. But not to demonstrate productivity or some other arbitrary metric(s). I self-cite because the images I use in my research are really difficult to make and I'm in a niche enough field that using the images really helps people who are not familiar with the area understand what is going on. These images were arduous to make, so I reuse them. (I should note, that I've gotten in "trouble" for reusing them without a citation).
Self-citation or clique-citation is only problematic if it doesn't fulfil its main purpose of providing relevant references. This should not be a surprising statement. But we just read an article where that property of citations is a sideshow. We got to the point where citation scores are bandied about without even asking the main question: "Well are the papers any good?" And now I worry that papers could get worse simply because of a citation-inflation.
I'm currently reading "The Systems Model of Creativity; The Collected Works of Mihaly Csikszentmihalyi" and was really surprised at the rate of self-citation in the included papers. Now it does seem to me that the cited studies are relevant. And I can't judge whether there would have been papers from other authors even better suited for citing.
And why am I even reading that book? Well because of the persistence with which Csikszentmihalyi gets cited in other writings I read. How do I know these writers weren't shills for Csikszentmihalyi? I don't care all that much when the material is good.
So in the end should I care about backwater publications that cite themselves excessively? Because I don't have to read them. As a consumer I don't seem to get hurt by the practice.
> The rate of duplication in the rest of the biomedical literature has been estimated to be between 10% to 20% (Jefferson, 1998), though one review of the literature suggests the more conservative figure of approximately 10% (Steneck, 2000). https://ori.hhs.gov/plagiarism-13
If work by another author was enough to inspire you and add a reference, then your own previous work should certainly qualify, if it added inspiration to the current paper.
(Like PageRank, it is very possible to discount internal PR/links under external links, and when you also take into account the authority of the referencer, you avoid scientists accumulating references from non-peer reviewed Arxiv publications).
reply