Yarrow Bouchard 🔸

1500 karmaJoined Canadastrangecosmos.substack.com

Bio

I got interested in effective altruism back before it was called effective altruism, back before Giving What We Can had a website. Later on, I got involved in my university EA group and helped run it for a few years. I joined the Effective Altruism Forum to try to figure out where effective altruism could fit into my life these days and what it means to me. You can read my latest thoughts on effective altruism here.

I write on Substack, and used to write on Medium.

Pronouns: she/her or they/them. 

Sequences
2

Criticism of specific accounts of imminent AGI
Skepticism about near-term AGI

Comments
757

Topic contributions
13

Thanks, Josh. If Anthropic completes its IPO by then, that would be a convenient way to get reliable revenue reporting. Let me double check the math later when I have a chance and confirm. I want to make sure I did the extrapolation correctly.

Suggestion: Leverage Research deep dive

Someone (other than me) should write a deep-dive post about the cult Leverage Research and its infiltration of effective altruism.

The story, in brief:

  • Leverage Research is a cult.
  • Leverage Research organized the first EA Summit in 2013 and the second EA Summit in 2014. The EA Summits were the first effective altruism conferences of any kind.
  • Leverage Research also helped to organize the first EA Global conferences, which began in 2015 and continue to this day.
  • In 2016, a major EA program, the Pareto Fellowship, was run largely by Leverage Research. There is some evidence the Pareto Fellowship was run in a cult-like fashion.
  • Leverage Research eventually gained full control of the Centre for Effective Altruism in 2018 when one of its members, Larissa Hesketh-Rowe, became the CEO. 

The purpose of the deep-dive post would be for people in EA to understand the truth about what happened. And to learn whatever lessons they think they should learn from that.

These are the questions I would recommend asking and attempting to answer in the deep-dive post:

  • Is Leverage Research a cult?
  • Did it take over the Centre for Effective Altruism?
  • Did it organize the EA Summits and the Pareto Fellowship? Did it play an important role in organizing the first EA Globals?
  • If so, how could the EA movement, particularly the core international leadership, let this happen?
  • If so, what might be the broader ramifications of this for the EA movement?
  • What (if anything) is there to learn from this?

I don’t know what the chances would be of actually getting funded, but someone who wanted to spend a lot of time investigating this topic could apply for a $1,000+ grant from the EA Infrastructure Fund

I’m not sure if Coefficient Giving (formerly Open Philanthropy) would even consider funding something so small and so specific to EA community self-reflection, but you can look at the relevant info here.

The bet would only be for a nominal amount of money (e.g., $20) to the charity of the winner’s choice. The purpose of the bet is not to make money but for people to publicly state and commit to specific, dated predictions with firm resolution criteria.

This is the same philosophy behind longbets.org (a project of the Long Now Foundation), whose winnings all go to the charity of the winner’s choice. The minimum you can bet there is $200. Long Bets has been around since 2002, and the first bet is about AGI: https://longbets.org/1/

Bets for large sums of money not for charity are not legally enforceable (and possibly, but not necessarily, technically illegal, depending on jurisdiction), so the money involved is always theoretical anyway.

Of course, the terminology I’m actually advocating is EA 1.0 and EA 2.0. I noticed that Daniel Frank (a board member of Giving What We Can Canada) already used this terminology in a post back in 2022. I also noticed one or two examples of other people using it organically.

To be clear, I didn’t come up with this terminology myself.

I think it should be fairly uncontroversial to say that the late 2000s, early 2010s version of EA focused almost exclusively on global poverty can be called EA 1.0. And that the late 2010s and 2020s version of EA focused largely on AGI safety, longtermism, and existential risk can be called EA 2.0. I think people generally agree that an important change happened, regardless of whether they think the change is good or bad.

Thanks.

I assume everyone who has donated to Stop AI, supported it, or been involved was shocked and horrified by Sam Kirchner’s threats to murder people. I can’t imagine anyone supported that.

Nothing I said was intended to imply that anyone who has supported Stop AI supported Sam Kirchner’s threats. It was just to establish that there is a meaningful connection between Stop AI and EA — that Stop AI is “in EA’s orbit”. 

I don’t believe Stop AI expelled Sam Kirchner. I think he resigned. After Sam Kirchner threatened to commit a mass shooting at OpenAI, a few people on the EA Forum expressed concern about what Stop AI said about it.

Someone should write a separate deep-dive post about Leverage Research and its infiltration of EA. (Although you’d be able to get a lot the same information just by reading the sources I cited.) A lot of people in EA are unaware of this story, but it’s important. It’s not exactly super secret or hidden — there seems to be general agreement that Leverage Research is a cult and the Centre for Effective Altruism describes its history with Leverage Research on its website under its “Mistakes we’ve made” page — yet people in EA don’t talk about it.

Maybe you could investigate the topic for yourself and write a deep dive post on it. Maybe someone else could.

The key thing is for people in EA to understand the truth about what happened. And to learn whatever lessons they think they should learn from that.

I’m a bit numb to procedural or formalistic critiques at this point because they always seem to just end up being another way to express substantive disagreement. It’s not like the EA community actually follows the writing and sourcing norms you’re advocating here.

The important thing is the substantive questions:

  • Is Leverage Research a cult?

  • Did it take over the Centre for Effective Altruism?

  • Did it organize the EA Summits and the Pareto Fellowship? Did it play an important role in organizing the first EA Globals?

  • If so, how could the EA movement, particularly the core international leadership, let this happen?

  • If so, what might be the broader ramifications of this?

  • What (if anything) is there to learn from this?

Similarly:

  • Are there multiple cults (or cult-like groups) that have emerged from or been associated with the LessWrong community (rationalist community)?

  • If so, why?

  • Is the threat of a Stop AI co-founder to commit a mass shooting a concern for EA?

  • If so, what could EA do in response? For instance, what could EA do to discourage extremist beliefs and behaviour, particularly violent behaviour?

  • Does any of this point to deeper concerns with EA? Is it just a string of bad luck, or is there something more here?

  • If it’s not just bad luck, what could be done better in the future?

Yes. It could just be a coincidence. But I don't think it's just a coincidence. I think the lack of critical thinking and lack of good norms you need to let a cult take over your movement is a bad sign, and may predict bad things to come. Particularly if there isn't an adequately large post-mortem, an adequate attempt to reckon with such a catastrophic mistake, such a catastrophic failure, and learn deep lessons from it.

This risks going down a rabbit hole, but I think one of the systemic problems with the EA movement is a consistent failure to do good post-mortems. (And while we're at it, good pre-mortems would probably help too.) Many movements, organizations, groups, individuals, etc. experience big failures. Sometimes they learn from them and turn things around. Sometimes they don't. There are several instances such as the Leverage Research infiltration, the Manifest racism scandal, and even the collapse of FTX where I think EA had opportunities to learn hard lessons and do things better in the future, but didn't.

But, of course, I can't prove any of this. It's all just my opinion.

But longtermism doesn't necessarily include AI safety, since many advocates of AI safety are not longtermists. If you think there's a 50%+ chance of superintelligence within a decade and a 5%+ chance of human extinction if superintelligence is created, you don't have to be concerned at all with anything that might happen 1,000+ years from now to treat that as an urgent priority.

In 2021, Will MacAskill, who coined the term longtermism, defined longtermism like this:

Longtermism is the view that positively influencing the longterm future is a key moral priority of our time.

In the introduction to the 2025 anthology Essays on Longtermism, Hillary Greaves, Jacob Barrett, and David Thorstad cite that definition from Will MacAskill. They also characterize longtermism like this:

A cluster of ideas going under the label ‘longtermism’ hold that considerations of the far future—on timescales of thousands, millions, or even billions of years—are highly significant for today’s decision-making.

Not all AI safety advocates are longtermists. Some are concerned with what will happen within the next 100 years and don't really think or care that much about the future 1,000+ years from now.

To accurately break down the EA 1.0 vs. EA 2.0 distinction by cause area, it would have to be something convoluted like: global health and development + animal welfare vs. AI safety + longtermism.

Load more