In a post entitled, “How Institutional Failures Undermine Trust in Science: The Case of a Landmark Study on Sustainability and Stock Returns,” Andy King (my collaborator on the project on scheduled post-publication review) tells a disturbing story of the failure of the scholarly publication process:
For a long time, I [King] resisted the accumulating evidence that our institutions for curating trustworthy science were failing.
I believed our academic gatekeepers–editors, reviewers, and research-integrity officers–were quietly doing their jobs. Overstretched, but nevertheless, curating a trustworthy scientific record and correcting it when problems appeared.
That belief ended when I attempted to replicate an extraordinarily influential article “The Impact of Corporate Sustainability on Organizational Processes and Performance,” by Robert Eccles, Ioannis Ioannou, and George Serafeim. The paper has been cited more than 6,000 times. Wall Street executives, top government officials, and even a former U.S. Vice President have all referenced it.
Uh oh . . . I have a horrible sense that I know what’s coming next:
It contains serious flaws and misrepresentations.
The article appeared in a prestigious journal, Management Science. The authors work at highly reputed institutions. As a result, I thought correcting the record would be straightforward.
I [King] ran into barrier after barrier.
OK, that doesn’t surprise me. I’ve had this sort of experience over and over. As the saying goes, it’s too hard to publish criticisms and obtain data for replication.
King continues:
The authors ignored me, the journal refused to act, and the scholarly community looked the other way. Two universities disregarded evidence of research misconduct–even after the authors admitted publishing a misleading report.
The article remains largely uncorrected–misleading thousands of people each year.
I believe our systems for curating trustworthy science are broken and need reformation.
Yup.
And now for the gory details:
The Authors
On September 11, 2023, I [King] emailed Eccles, Ioannou, and Serafeim to explain that I was attempting to replicate their study and had encountered serious problems:
• The reported method did not work as described.
• A key result seemed to be mislabeled as statistically significant when it was not.
• Some measures defied construction.
• Critical statistical tests appeared to be missing.
• The sample was highly unusual.
I explicitly acknowledged uncertainty and asked for help. Over roughly half a dozen follow-up emails, I shared progress updates and offered to collaborate.I received no response.
My experience is not unusual. Bloomfield et al. (2018) show that requests from replicators are often ignored, delayed, or deflected. Because published articles frequently omit key details, authors can block replication simply by refusing to engage.
The Community of Scholars
I turned to colleagues and respected scholars for advice. I asked for help encouraging the authors to engage. I emphasized that mistakes happen–my own work is not unblemished–and that correcting errors strengthens, rather than diminishes, scholarly standing. I heard:
• “I can’t do anything–it would cause conflict.”
• “Your email is too long.”
• “I’m underwater for the next month.”
• “I’m too much of a coward.”
The last came from an internationally respected scholar with a chaired position at a top university. [Don’t worry, that wasn’t me — AG] I [King] appreciated the candor. It revealed an uncomfortable truth: much of social science operates on a culture of go-along, get-along.“Once a paper is published… it is more harmful to one’s career to point out the fraud than to be the one committing it” (a different Bloomfield et al., 2018, link).
The Journal
Having received no response from the authors, I contacted Management Science. After getting advice, I submitted a comment.
It was rejected.
The reviewers did not address the substance of my comment; they objected to my “tone”.
Ahhhh, the tone police!
King continues:
They told me that published authors should be granted “discretion” in conducting their work and that replicators should tread very lightly. One reviewer was “inclined to turn down any invitation to review a revision” unless it was accompanied by a note from the original authors.
Knowing such a note would never come, I appealed. Rejected. I appealed again. Rejected.
The authors did admit to the editor that they had misreported a key finding–labeling it as statistically significant when it was not. The authors claimed the error was a “typo.” They intended to type “not significant” but omitted the word “not.”
Oh, I hate when that happens! So frustrating how the typos always seem to support the overblown claims being made.
King continues:
They did not address the implications of this “typo”–that it misrepresented the evidence for a central claim of the paper, that corporate sustainability increases stock returns.
I asked the journal to correct the record. Rejected.
My experience is not unusual. As one respondent told Bloomfield et al. (2018): “Replication studies don’t get cited, and journals don’t publish them. Nor do people get promoted for replication studies”.
The good news is that King and I are both too old to worry about getting promoted.
King continues:
Help from Outsiders: LinkedIn and an Upstart Replication Journal
I decided I needed to go outside the standard process and post publicly about the “typo” on LinkedIn.
Days later, I heard that the journal would publish a correction.
I was told the authors had submitted the correction before my post, but it had been misplaced and forgotten.
I believe the journal’s new editor found this news to be as incredible as I did. He quickly published an erratum.
I also submitted my replication to the Journal of Management Scientific Reports (JOMSR). This upstart publication was started in 2022 by a small group of courageous scholars who wanted to provide an outlet for replication studies like mine. I was impressed by their thorough reviews and tough guidance.
In spring 2025, JOMSR published my replication study.
Research Integrity Offices (Part 1)
While revising my replication for publication, I became convinced of a more serious issue: the method reported in Eccles, Ioannou, and Serafeim (2014) was not the method actually used. Worse, the true method could not support their “findings”.
I contacted the authors again. No response.
I decided a research integrity complaint was in order.
In July and August 2025, I submitted complaints to Harvard Business School and London Business School. I alleged that the reported method could not have been conducted as described–and that the results were therefore uninterpretable.
(A technical aside describing the study’s method may be useful here. Feel free to skip.)
• The empirical strategy in Eccles, Ioannou, and Serafeim (2014) rests on a demanding requirement: the “treated” and “control” firms must be so closely matched that which firm is treated is essentially random. The authors appear to recognize this, reporting that they used very strict matching criteria “to ensure that none of the matched pairs is materially different.”
• Despite their strict criteria, they also claim to have achieved remarkable success in finding precise matches, reporting that 98% of their “high sustainability” firms could be matched with a near-twin “low sustainability” firm. Yet when I attempted to replicate the study, I achieved a much lower match rate–fewer than 15%. To better understand the discrepancy, I conducted a probability analysis using a Monte Carlo simulation. I determined that the reported matching success was highly unlikely–many, many, many times less than winning the lottery.
• Either their matching process was precise, in which case they would not have enough pairs to run their analysis, or it was loose, in which case their analysis could not be interpreted.
(End of aside.)Shortly after I submitted my complaint, the authors acknowledged they had misreported their method.
But they did not ask Management Science to correct the text of their article.
Research Integrity Offices (Part 2)
Eccles, Ioannou, and Serafeim explained that the misreport was an unfortunate accident. There had been two studies, they said, and the false description belonged to an “exploratory” study that was later removed to satisfy length requirements, except the sentences describing its matching process, which were inadvertently left behind. As a result, those sentences now appeared to describe the “main” analysis, but that is not what they had intended. It might look like misrepresentation, but it was just an editing error.
They did not explain that this meant all of their results were uninterpretable.
The explanation also conflicts with the record.
• The incorrect claim appears in the earliest available draft of their article–marked “NEW!” on HBS’s site.
• Over several later drafts, the false claim was retained and even edited, rather than removed.
• The “exploratory study” does not appear in any available draft.In light of these inconsistencies, I submitted a revised complaint to Harvard Business School and London Business School.
Harvard Business School responded: “Whether or how the School does or does not move forward… will not be communicated to you.”
LBS was more open and responded quickly, concluding that the false claim was not an “intentional falsehood”. Why? Because the LBS professor (Ioannou) “did not have access to the raw data and did not conduct the analyses in question.”
That’s technically known as the “Ariely defense.” You’re the author of the paper but you didn’t touch the data, therefore you couldn’t possibly have cheated.
And then we get something we’ve heard many, many times before:
And in any case, the problem was of a “minor nature”, apparently because it pertained to some other study and thus did “not impact the main text, analyses, or findings.”
It’s funny how removing these fraudulent or erroneous analyses never affect the main conclusions of the study. It kind of makes you wonder why they went to the trouble of gathering and analyzing the data at all!
King continues:
Sadly, LBS’s response is empty.
• Data access is immaterial. I did not allege data fabrication.
• The false claim is not minor. It is the difference between a usable and useless study.
• It does not address the central question: Did the exploratory study ever exist? If not, false statements were published twice–first in the article, and then in the offered explanation.LBS did conclude that the author engaged in “poor practice”, which they planned to address through “education and training or another non-disciplinary approach.”
I suggest LBS begin by explaining an author’s duty to correct errors in published work.
Where This Leaves Us
Eccles, Ioannou, and Serafeim (2014) remains only partly corrected in the pages of Management Science. Diligent readers may discover the erratum correcting the “NOT significant” finding, but they will not learn of the misreported method in the pages of Management Science. Thus, thousands of readers remain misled.
Our institutions for curating trustworthy social science are not working. They must be changed, reformed, and revitalized.
What you can do
1. Stop citing single studies as definitive. They are not. Check if the ones you are reading or citing have been replicated.
2. If you or someone else finds an error in your published work, publish a correction.
3. If one of your colleagues is behaving unprofessionally, tell them to stop.
4. Support replication. Encourage others to do so. Support the Journal of Management Scientific Reports.
5. Find out about the research integrity policies at your institution. If they are weak, strengthen them.
6. If you know Eccles, Ioannou, and Serafeim, ask them to retract their article, or at least publish another correction.What else needs to change
For years, I studied industry self-regulation. The evidence is clear: it works only when it is transparent, independently monitored, and supported by graduated sanctions. Applying this to the curation of science.
1. Journals should disclose comments, complaints, corrections, and retraction requests. Universities should report research integrity complaints and outcomes.
2. An independent third-party should audit the process.
3. Penalties should reflect the severity of the violation, not be all-or-nothing.
4. And to ensure the system works, we need what Andrew Gelman and I call FurtherReview.
Let me just add one more thing.
I don’t know any of the authors of the paper under discussion–indeed, I’d never known of them, or their paper, before hearing this story from King–so I’m speaking in general terms:
– Whether or not the authors were lying or intentionally misrepresenting at any point, I agree with King that, based on the evidence above, they did research misconduct.
– This doesn’t mean that the authors of that paper are bad people!
We should distinguish the person from the deed. We all know good people who do bad things, indeed I’ve received some speeding tickets in my time, and there are lots of good people who’ve done worse than that. I’ve been in the car with some drunk drivers, some dangerous drivers, who could easily have killed people: that’s a bad thing to do, but I wouldn’t say these were bad people. They were just in situations where it was easier to do the bad thing than the good thing
What Eccles, Ioannou, and Serafeim did is much less bad than my friends driving drunk, but it’s still bad, but the same principle applies. They’re living in a world in which doing the bad thing–covering up error, refusing to admit they don’t have the evidence to back up their conclusions–is easy, whereas doing the good thing is hard.
OK, actually doing the good thing is easy. You just admit your error. I’ve done it myself–it’s super-easy, you just contact the journal and write a short, direct, and honest correction, and they’ll publish it. But to lots of people, it seems hard. As researchers they’ve been trained to never back down, to dodge all criticism. I don’t like what they did, but I imagine that they view their actions as something like how I might view a speeding ticket: yeah, I shouldn’t have done it, but it happens in the past.
From that perspective, the real problem is not the sin but rather the mistaken attitude that, in science and scholarship, what’s past is past. There’s a horrible sort of comfort in thinking that whatever you’ve published is already written and can’t be changed. Sometimes this is viewed as a forward-looking stance, but science that can’t be fixed isn’t past science; it’s dead science. And what bothers me about Eccles, Ioannou, and Serafeim, and all the many error-deniers like them, is that they don’t seem to realize this. It’s this fundamental misunderstanding of the scientific and scholarly endeavor, more than the dishonesty or sloppiness or whatever is the specific unethical behavior, that bothers me.
But, yeah, Andy King has a point that when universities, journals, and other institutions support the bad behavior, that’s not good. That doesn’t help at all. In all seriousness, you gotta feel a little sorry for Harvard Business School: they’ve had so many of these scandals now. It’s not like Duke and MIT business schools, which just had one scandal each–actually it was the same scandal for the two of them.
“Stop citing single studies as definitive. They are not. Check if the ones you are reading or citing have been replicated.”
I guess I’m just naive but I have a hard time believing that a serious researcher would consider one study as definitive (on anything that’s even mildly complicate).
From my experience in social science, including some experience in managment studies specifically, researchers regularly belief things – and will even give policy advice based on those beliefs – that have not even been seriously tested, or have straight up been refuted. Especially when it fits their prior and/or preferred narratives and/or when it’s just a nice story (I guess ‘companies that do csr stuff outperforming those that don’t’ ticks all those boxes for a lot of people). In that sense, a single study is already a strong basis, comparatively speaking, depressing that may be. Agreed that serious researchers wouldn’t do that, though.
I think there sometimes is a belief that the last study is the ‘definitive’ one, for example when it corrects a limitation of a previous study. Of course, it still is only one study.
I can understand why a study that corrects a previous study might be seen as *more* definitive than it’s predecessor, but I really can’t unsersnd why anyone wouldn’t regularly apply the “one study rule.”
Books and magazine articles presenting behavioral science to the public do it all the time.
That doesn’t make it right. Popular social “science” is even worse than social “science.”
Often single studies are definitive in social sciences because if a paper is published for a result “we find X” (e.g., “we find that green jelly beans make you a better worker”), then no follow-up studies will be published because they will lack novelty. “We already know that green jelly beans make you a better worker!” => Rejected.
To get published, you will need something new (“we find that red jelly beans make you more optimistic” or “contrary to earlier study, we find that green jelly beans make you a worse worker” … a null result usually won’t do).
The better approach for non-researchers is probably to ignore almost all p-hacked social science for the bunk that it is (absent pre-registration, etc.).
Am I missing something obvious? The Bloomfield et al. 2018 paper doesn’t seem to contain the quotes that King includes.
There are two such papers cited (the second is related to the first). Here’s the link (appears on page 92 (95 by pdf count)) https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3119833
The citation is linked correctly in the linked in post (https://www.linkedin.com/pulse/how-institutional-failures-undermine-trust-science-andrew-king-2d2ue accessible without a login).
(I haven’t looked at the papers, only checked for the quote in response to your comment).
Might have been my error. You need the version that is “free responses”. Sorry.
Anon:
My bad. There were two different Bloomfield et al. (2018) links in King’s post, and I hadn’t included both of them. I fixed it.
thanks!
I wish the incentives were as you described them, but there’s a reason why these things keep happening in business schools.
In political science (and hopefully other fields), you get respect by being seen as a thoughtful researcher with integrity. If someone produces new research that challenges your own (or even if you just make an outright error), the best way to regain the field’s respect is to acknowledge and admit the criticism.
This incentives don’t work that way in business schools, where career success depends upon creating a clear “brand.” People do not care about science or good research, they care about being known for something specific. So in the case of the junior author on EIS, his career has been built entirely on being “guy who has shown that corporate sustainability is profitable” rather than “guy who does good work on corporate sustainability.”
Plus there are (bad) outside incentives that exist in business schools. As the word “brand” suggests, there are also very lucrative outside options to be gained from telling people something that they want to hear (“sustainability is profitable!”) and very little profit to be made from telling people something inconvenient (“sorry folks, there is no clear relationship between sustainability and profitability, if you want to be more sustainable you’ll have to find some other argument to convince your shareholders”).
Minor point – The text after you mention the Ariely defense isn’t italicized, so it’s a little difficult to tell your words apart from Dr. King’s.
Fixed; thanks.
I believe that “Ariely defense” should be written in Calibri.
Comic Sans.
This lines up with two frauds at the Journal of Accounting Research. In one, the authors apparently edited a small number of observations to obtain results. After years of back and forth in Econ Journal Watch, the journal did nothing.
In another, nearly identical to the case in the post, the authors grossly misrepresented their data. While they did not fabricate data, the data they used was irrelevant to the research question and generated entirely meaningless statistics and inferences. The authors were aware they were not using the data they claimed to use. They simply lied about it. Imagine conducting a study about birds, but actually using data about bears and just saying that bears are birds (they both start with B, close enough, right?). Again, the journal did nothing. Conveniently, the data is proprietary so no one can obtain the correct data to conduct a real study. Interestingly, one of the authors on this fraudulent study is King’s colleague, and King’s university has no research integrity office. The other author manages the corrupt journal!!
I could not identify the second case. If you wish, send more detail to aaking@bu.edu
I am not sure what study you are referring to. For the record, BU does have a research integrity process and officer (Kate Mellouk) — but schools believe that they can only investigate cases were the research was performed by a faculty member while at the school. This limits who can investigate the infamous “Sign on Top” study by Lisa L. Shu, Nina Mazar, Francesca Gino, Dan Ariely, and Max Bazerman. Only HBS can investigate Shu, Gino, Bazerman. Only Toronto can investigate Mazar (now at BU), and only Duke can investigate Ariely.
I think this is a BIG part of the problem. Strategies for fraudsters: do a lot fast and then move to a new school.
Quote from above: “I think this is a BIG part of the problem. Strategies for fraudsters: do a lot fast and then move to a new school.”
Interesting comment to me in light of what I wrote in a manuscript mentioned somewhere else in the comment section (I used “Anonymous” in the comment there because I usually do, but will use “AAAnonymous” now because I am replying to another “Anonymous”). This is what I wrote in the manuscript which I was reminded of when reading your comment just now:
“The psychopathic psychological scientist is irresponsible, unreliable, and unconscientious (cf. Cleckley, 1941/1988, p. 340-341; Miller & Lynam, 2015). The psychopathic psychological scientists disregard their (arguably) professional obligations and responsibilities concerning things like student supervision, citing sources, and performing peer review (cf. Gopalakrishna et al., 2022; Schneider et al., 2024). This kind of unreliable and irresponsible behavior is however masked or alternated by situations or time periods in which the unreliable behavior is not present (cf. Cleckley, 1941/1988, p. 340-341). This makes it possible that the unreliability and irresponsibility of the psychopathic psychological scientist largely goes unnoticed, which may even be facilitated by certain features of present-day academia and psychological science such as frequent changing of place of work and collaborators (cf. Levelt et al., 2012, p. 37-46), or short-term international mobility (cf. Haupt, 2021).”
There is an alternative publication model available. Physics has the Arxiv already, but there is a new open-access journal called the Science Post, which has no publication fees and which makes all referee reports public. It is run by scientists and is nonprofit. It is also not restricted to physics.
The major impetus for this project was not misconduct or fraud–it was excessive and extortionate fees levied by for-profit organizations like Springer, Elsevier, and even the journal Science (author and reader fees). However, this would decrease misconduct as well.
The University of California system briefly ended its contract with Elsevier due to excessive costs (https://www.universityofcalifornia.edu/news/why-uc-split-publishing-giant-elsevier). Hopefully more will do the same.
https://scipost.org/about
I don’t have much of substance to add to this story, except to applaud Andy for a thorough and principled approach to a pretty galling issue.
It’s tempting to ascribe all of this to management science (I resisted the temptation to put scare quotes around “science”), but it’s likely the case in plenty of other disciplines like psych, sociology, health, etc.
Don’t look too closely at the ESG literature. After investors shifted to low-cost index funds, there was a need to justify some sort of active management and the correspondingly higher management fees. ESG was a good candidate because it spoke to people’s concerns (it worked for recycling) and who doesn’t want to believe that doing the socially responsible thing would have some kind of financial return? You don’t need to be a genius to realize what kind of findings will get you lucrative speaker fees (which don’t need to be disclosed as conflicts of interest) and which get you shunned.
This all reminded me the following paper titled “Scientific fraud and the power structure of science” by B. Martin (1992). A quote:
“Several of the common misrepresentations and biases are natural outgrowths of the hierarchies within scientific organisations: misrepresentation in citations, false pictures of research in grant applications, appointments of cronies and exploitation of subordinates.Many of those who rise within the hierarchy do so by claiming an excess of credit for their own contributions; once somewhat up the hierarchy, it is easier to use the power of position to continue the process. It is easy to see why many of these practices are standard: they serve the interests of the more powerful members of the research community.” (p. 89).
And the following might also be relevant and appropriate to quote in light of this all. From a paper by Biagioli et al. (2019) titled “Academic misconduct, misrepresentation and gaming: A reassessment”:
“In an era characterized by an emphasis on measuring academic performance, there has been a proliferation of scandals, questionable behaviors and devious stratagems involving not only individuals but also organizations, whether it be universities ‘fine-tuning’ data to score well on global university rankings, or editors and reviewers engaging in coercive citation and other abuses, or journal publishers creating so-called ‘predatory’ or ‘pseudo’ journals, or conference organizers offering ‘pay-to-play’ conferences and guaranteed publication of the presenters’ abstracts.”
“What we are apparently witnessing now is far more metrics-oriented misconduct or questionable behavior aimed not so much at producing a high-profile publication but rather at incrementally increasing an individual’s or organization’s reputation, for example, through self-citation or departmental faculty members citing each other, sometimes with the connivance of their university managers.”
I encountered the Martin (1992) paper, and the Biagioli et al. (2019) paper, when writing a manuscript in which I used a paper by Kwok (2005). I find the paper by Kwok (2005) more and more interesting and possibly useful the more I read about problematic issues in science and academia. Kwok (2005) focuses on problematic coauthorship and publication issues, and mentions a particular type of “serial abuser” who uses his experience and deviousness to exploit uncertainties or ambiguities in research guidelines and prospers in poorly regulated, grey areas. (see p. 554). Kwok (2005) notes that:
“Finally, the personality profile of the scientific fraudster is a largely ignored area of study. More research is required on the personality and psychological aspects of scientific misconduct, and the sociology of scientific fraud.” (p. 555).
I attempted to something with these quotes from Kwok (2005), and other sources and papers, and came up with the following personality profile which might be something to keep in mind, or focus on, or do research on, concerning several problematic issues in science and acedemia, and concerning individual scientists and their behavior.
• High social status, absent or limited aggressive antisocial behavior, and possibly high intelligence
• Desire to have power and status
• Competitive world-view and hyper-competitive attitude
• Weak moral identity and low moral standards
• Selfish, egocentric, and lacking in empathy
• Narcissistic, possibly grandiose
• Bold
• Superficially charming
• Mean
• Untrustworthy, deceitful, and manipulative
• Irresponsible, unreliable, and unconscientious
• Lacking in remorse
These fit both Donald Trump and Elon Musk perfectly. I would add: childish, petulant, and vindictive to the list.
The fundamental problem is, indeed, institutional: 1) Generally we have come to believe that a “good” paper is has to confirm a hypothesis in a “topic of interest” (good research that goes against a “topic of interest”, a less “interesting” topic, etc. is less likely to be published with more scrutiny) so this skews a lot of researchers to find some kind of supporting result in said topic (which pushes pushes researchers to this behaviour). In this case it is no secret that sustainability was a topic of interest that, especially business schools, wanted to push. 2) Like you described there is no incentive at all to be critical of existing research that has been published, and it would actively harm ones professional career and social capital (which is heavily linked to ones professional career in academia) and there should be a (may it be anonymous) system that rewards finding mistakes or correcting existing research (which is a, in my opinion, an incredible scholarly contribution that is on par with writing a research paper yourself). Replicating papers should be rewarded but it’s not – many things can be done like getting co-authorship in the correction for finding a major mistake, monetary bounties for research in respectable journals, etc. 3) Institutions themselves tend to protect their researchers and sweep things under the rug to avoid a whiff of embarrassment. Like you said, mistakes happen and things should get settled fast with no-BS. I also agree with the lack of accountability that you mention and it is no secret that there are a lot of RAs doing the data work – who also have their own incentives to provide supporting results – and coauthors should have full responsibility/accountability on what happens here.
I am not surprised this paper is not replicable. Even if it were not replicable, it is methodologically bankrupt and an embarrassment.
Looking only at their stock return methodology, there are enough glaring flaws to warrant a desk rejection at a good finance journal:
1) Their stock return results likely result from survivorship bias.
The authors condition their high and low sustainability samples on survival to the present day. They then assert that, because this selection applies to both samples, it induces no bias.
This is true, of course, if the sample sorting is perfectly random. But it is not. Selection acts in surprising ways in return studies. As an example, if I pick two samples with differing volatility **and condition on survival**, the high-vol companies have higher expected returns. You are picking the right tail of a more dispersed distribution.
And, in fact, the authors note in the text that the “High” sample is higher vol than the “Low” sample, exactly the direction needed to produce this bias. (This is consistent with their Table 7, which shows that the “High” sample has higher beta, which is highly correlated with vol, than the “Low” sample.)
The survivorship conditioning is so weird in part because it is non-standard. The natural and usual choice is not to condition on survival. We know what the P&L is for holding firms that go bankrupt or are acquired — the finance literature deals with this reality all the time. Survivorship conditioning is a third rail precisely because it has these surprising and subtle effects.
2) Their alpha test in Table 8 is non-standard and just plain wrong.
Don’t believe the stated significance level for the intercept in Table 8. Note that the number of observations is 180 — 90 high firms + 90 low firms. That’s right, they are treating each firm as an independent (more or less) observation. Hard to express how taboo or bad this is in the context of the finance literature. The reason is that contemporaneous firm returns are highly dependent. The authors’ standard errors do not (cannot) correct for this form of dependence. They are grossly, grossly, understated. (One correct thing to do, approximately, would be something like treating firm-years as observations and computing clustered standard errors that are robust to contemporaneous correlation.)
This is not a fine point. Since at least 1973 financial econometrics has been refining and re-refining ways to make correct inferences about expected returns given dependence patterns in the data.
3) Rather than considering the data and selecting a methodology for picking “High”, they actually **go and interview firms** to understand the history, and exclude some firms on that basis. If ever there were a way for bias, conscious or unconscious, to creep into a study, this is it.
4) Then there is the latent notion of sustainability as an experimental treatment. Never explicitly stated, but implied in how they interpret the results. In reality, the firms who chose to emphasize sustainability were fundamentally different across any number of non-measured axes. If, and this is a big if, we buy the return results, who is to say these results come from the sustainability decision itself, and not from any of the factors that drive such a decision? Such as which social network the CEO and Board are in, party affiliation, etc. We already know that these firms were higher vol firms, for example.
That’s all I saw in a short read.
None of this is too surprising — it would be more surprising if the paper *were* convincing! Management Science is not where serious finance papers are published. And none of these authors has the background to be doing this kind of work. I would be surprised if any finance academic or industry quant looked at this abstract and changed their beliefs in the slightest. They see it as the grift that it is.
On the grift point, this paper needs to be understood in the context of the ESG “movement”. In 2018, every asset management firm (from sad personal experience) was desperately torturing the ESG data to find anything — anything — that predicted historical returns. Even setting aside the multiple comparisons problem, which of course we all did, there was absolutely nothing (ex-post) predictive in the data. And yet we all knew that pensions and endowments were really hungry for ESG, and desperately wanted the result that “ESG is good business.” This led to bad research both inside and outside of industry.
As some evidence of this hunger, note that partially on the back of this strain of work, State Street partnered with one of the authors to promote ESG, and KKR and BCG partnered with another. The third is working with DWS on ESG. And that is just a sample.
So there we are. That is the real reason that this appalling work can’t be pulled back. The authors are in deep. The firms they have partnered with are exposed if their ESG experts lose face. A retraction might even lead to the authors being sued.
You will never, ever, see a glint of an admission of anything from these three.