News and links

• Ray Arnold provides a useful list of ways the average person help with AI safety.
• New from OpenAI: attacking machine learning with adversarial examples.
• OpenAI researcher Paul Christiano explains his view of human intelligence:

  I think of my brain as a machine driven by a powerful reinforcement learning agent. The RL agent chooses what thoughts to think, which memories to store and retrieve, where to direct my attention, and how to move my muscles.

  The “I” who speaks and deliberates is implemented by the RL agent, but is distinct and has different beliefs and desires. My thoughts are outputs and inputs to the RL agent, they are not what the RL agent “feels like from the inside.”

• Christiano describes three directions and desiderata for AI control: reliability and robustness, reward learning, and deliberation and amplification.
• Sarah Constantin argues that existing techniques won’t scale up to artificial general intelligence absent major conceptual breakthroughs.
• The Future of Humanity Institute ran a “Bad Actors and AI” workshop.
• FHI is seeking interns in reinforcement learning and AI safety.
• Michael Milford argues against brain-computer interfaces as an AI risk strategy.
• Open Philanthropy Project head Holden Karnofsky explains why he sees fewer benefits to public discourse than he used to.

News and links

• An Asilomar panel on superintelligence (video) gathers Elon Musk (OpenAI), Demis Hassabis (DeepMind), Ray Kurzweil (Google), Stuart Russell and Bart Selman (CHCAI), Nick Bostrom (FHI), Jaan Tallinn (CSER), Sam Harris, and David Chalmers.
• Also from Asilomar: Russell on corrigibility (video), Bostrom on openness in AI (video), and LeCun on the path to general AI (video).
• From MIT Technology Review‘s “AI Software Learns to Make AI Software”:

  Companies must currently pay a premium for machine-learning experts, who are in short supply. Jeff Dean, who leads the Google Brain research group, mused last week that some of the work of such workers could be supplanted by software. He described what he termed “automated machine learning” as one of the most promising research avenues his team was exploring.

• AlphaGo quietly defeats the world’s top Go professionals in a crushing 60-win streak. AI also bests the top human players in no-limit poker.
• More signs that artificial general intelligence is becoming a trendier goal in the field: FAIR proposes an AGI progress metric.
• Representatives from Apple and OpenAI join the Partnership on AI, and MIT and Harvard announce a new Ethics and Governance of AI Fund.
• The World Economic Forum’s 2017 Global Risks Report includes a discussion of AI safety: “given the possibility of an AGI working out how to improve itself into a superintelligence, it may be prudent – or even morally obligatory – to consider potentially feasible scenarios, and how serious or even existential threats may be avoided.”
• On the other hand, the JASON advisory group reports to the US Department of Defense that “the claimed ‘existential threats’ posed by AI seem at best uninformed,” adding, “In the midst of an AI revolution, there are no present signs of any corresponding revolution in AGI.”
• Data scientist Sarah Constantin argues that ML algorithms are exhibiting linear or sublinear performance returns to linear improvements in processing power, and that deep learning represents a break from trend in image and speech recognition, but not in strategy games or language processing.
• New safety papers discuss human-in-the-loop reinforcement learning and ontology identification, and Jacob Steinhardt writes on latent variables and counterfactual reasoning in AI alignment.

News and links

• Gwern Branwen argues that more autonomous intelligent systems are likely to systematically outperform “tool-like” AI systems.
• “Policy Desiderata in the Development of Machine Superintelligence“: Nick Bostrom, Allan Dafoe, and Carrick Flynn outline ten key AI policy considerations.
• Faulty Reward Functions in the Wild: OpenAI’s Dario Amodei and Jack Clark illustrate a core obstacle to aligning reinforcement learning systems. 
• Open Phil updates its position: “On balance, our very tentative, unstable guess is the ‘last dollar’ we will give (from the pool of currently available capital) has higher expected value than gifts to GiveWell’s top charities today.”
• Carl Shulman argues that risk-neutral philanthropists of all sizes who are well-aligned with Open Phil should use donor lotteries to rival Open Phil’s expected impact per dollar.