Hi! This is Eliezer Yudkowsky, original founder but no-longer-moderator of LessWrong.com and also by not-quite-coincidence the first AI In A Box Roleplayer Guy.

I'm a bit disheartened that Randall Monroe seems to have bought into RationalWiki's propaganda on the Roko's Basilisk thing, but I'm aware that it's hard to find anything else on the Internet and that I've simply never had time to write up or try to propagate any central page to refute it.

RationalWiki hates hates hates LessWrong because... well, I'm not sure, but so far as I can tell, they think we're getting above ourselves by talking about probability theory when good skeptics are salt-of-the-Earth homeopathy-mockers who shouldn't sound so snooty. RationalWiki does not obey norms of normative discourse. They really really do not obey norms of rational discourse. Including norms like "not making stuff up".

RationalWiki basically invented Roko's Basilisk as a meme - particularly the meme that there's anyone out there who believes in Roko's Basilisk and goes around advocating that people should create AI to avoid punishment by it. So far as I know, literally nobody has ever advocated this, ever. Roko's original article basically said "And therefore you SHOULD NOT CREATE [particular type of AI that Yudkowsky described that has nothing to do with the Basilisk and would be particularly unlikely to create it even given other premises], look at what a DANGEROUS GUY Yudkowsky is for suggesting an AI that would torture people that didn't help create it" [it wouldn't].

In the hands of RationalWiki generally, and RationalWiki leader David Gerard particularly who also wrote a wiki article smearing effective altruists that must be read to be believed, this somehow metamorphosed into a Singularity cult that tried to get people to believe a Pascal's Wager argument to donate to their AI god on pain of torture. This cult that has literally never existed anywhere except in the imagination of David Gerard.

Checking the current version of the Roko's Basilisk article on RationalWiki, virtually everything in the first paragraph is mistaken, as follows:

Roko's basilisk is a proposition that says an all-powerful artificial intelligence from the future may retroactively punish those who did not assist in bringing about its existence.

Roko's basilisk was the proposition that a self-improving AI that was sufficiently powerful could do this; all-powerful is not required. Note hyperbole.

It resembles a futurist version of Pascal's wager; an argument used to try and suggest people should subscribe to particular singularitarian ideas, or even donate money to them, by weighing up the prospect of punishment versus reward.

This sentence is a lie, originated and honed by RationalWiki with the deliberate attempt to smear the reputation of what, I don't know, Gerard sees as an online competitor or something. Nobody ever said "Donate so the AI we build won't torture you." I mean, who the bleep would think that would work even if they believed in the Basilisk thing? RationalWiki MADE THIS UP.

Furthermore, the proposition says that merely knowing about it incurs the risk of punishment.

This is a bastardization of work that I and some other researchers did on Newcomblike reasoning in which, e.g., we proved mutual cooperation on the oneshot Prisoner's Dilemma between agents that possess each other's source code and are simultaneously trying to prove theorems about each other's behavior. See http://arxiv.org/abs/1401.5577 The basic adaptation to Roko's Basilisk as an infohazard is that if you're not even thinking about the AI at all, it can't see a dependency of your behavior on its behavior because you won't have its source code if you're not thinking about it at all. This doesn't mean if you are thinking about it, it will get you; I mean it's not like you could prove things about an enormous complicated AI even if you did have the source code, and it has a resource-saving incentive to do the equivalent of "defecting" by making you believe that it will torture you and then not bothering to actually carry out the threat. Cooperation on the Prisoner's Dilemma via source code simulation isn't easy to obtain, it would be easy for either party to break if they wanted, and it's only the common benefit of cooperation that establishes a motive for rational agents to preserve the delicate conditions for mutual cooperation on the PD. There's no motive on your end to carefully carry out necessary conditions to be blackmailed. (But taking Roko's premises at face value, his idea would zap people as soon as they read it. Which - keeping in mind that at the time I had absolutely no idea this would all blow up the way it did - caused me to yell quite loudly at Roko for violating ethics given his own premises, I mean really, WTF? You're going to get everyone who reads your article tortured so that you can argue against an AI proposal you don't understand? In the twisted alternate reality of RationalWiki, this became proof that I believed in Roko's Basilisk, since I yelled at the person who invented it without including twenty lines of disclaimers about what I didn't necessarily believe. And since I had no idea this would blow up that way at the time, I suppose you could even read the sentences I wrote that way, which I did not edit for hours first, if you supposed I was a complete gibbering idiot. And then, since Roko's Basilisk was a putatively a pure infohazard of no conceivable use or good to anyone, and since I didn't really want to deal with the argument, I deleted it from LessWrong which seemed to me like a perfectly good general procedure for dealing with putative pure infohazards that jerkwads were waving in people's faces. Which brought out the censorship!! trolls and was certainly, in retrospect, a mistake.)

It is also mixed with the ontological argument, to suggest this is even a reasonable threat.

I have no idea what "ontological argument" is supposed to mean here. If it's the ontological argument from theology, as was linked, then this part seems to have been made up from thin air. I have never heard the ontological argument associated with anything in this sphere, except on this RationalWiki article itself.

It is named after the member of the rationalist community LessWrong who most clearly described it (though he did not originate it).

Roko did in fact originate it. Also, anyone can sign up for LessWrong.com, David Gerard has an account there but that doesn't make him a "member of the rationalist community".

And that is just the opening paragraph.

I'm a bit sad that Randall Monroe seems to possibly have jumped on this bandwagon - since it was started by people who were in it to play the role of jocks sneering at nerds, the way they also sneer at effective altruists, and having XKCD join in on that feels very much like your own mother joining the gang hitting you with baseball bats. On the other hand, RationalWiki has conducted a very successful propaganda campaign here, with deliberately credulous cooperation from, e.g., Slate, which saw an opportunity to hit some associated targets of opportunity such as Peter Thiel, who was heavily suggested to be funding AI research on the basis of this idea, which to the best of my knowledge Thiel has never said one single word about ever; and it goes without saying that Slate didn't contact any of the parties smeared, such as myself, to check accuracy. So it's saddening but not too surprising if Randall Monroe has never heard hinted any version but RationalWiki's. I hope he reads this and reconsiders.