Design A Better Chess

By Robin Hanson · May 22, 2017 9:00 am · Comments (30) · ·

Friday the Wall Street Journal published my review of Garry Kasparov’s new book Deep Thinking. I end with:

I’ve always been a bit skeptical of the high status of chess champions, whom many consider intellectuals (rather than, say, sports stars). But in “Deep Thinking,” Mr. Kasparov has changed my mind. He praises Mikhail Botvinnik, the founder of the Soviet chess school where he trained, for practicing an “intense regime of self-criticism.” Chess champions are rewarded for brutal honesty about their habits and strategies. If only most tenured professors and business executives were this conscious of their limitations and blind spots.

“Few young stars in any discipline are aware of why they excel,” Mr. Kasparov writes. Like Mr. Kasparov, I don’t know why he was great. But I know now why I’m glad we have him. We need at least a few of our most celebrated minds to be this intellectually honest with themselves, and with us.

While all sports reward honesty and self-criticism on your sports performance, in more intellectual sports that honesty can more influence your opinions on more important topics. Which raises the question: can we design a game that promotes even more useful honestly? As I spent some of my youth doing game design, and had a friend who shared that interest, I know that designing games is hard; there are many relevant constraints of which most players are unaware (see the usual literature). For this game design task, all those usual constraints apply, and we must attend to some added criteria:

  • Relevant: We’d like the topics where this game rewards insight and understanding to be closer to the topics that matter, where brutal honesty would be more useful to the world.
  • Fair: Even with relevant topics, the game can’t seem to greatly favor people who by class or culture get much more direct personal info and experience regarding those relevant topics. Anyone should be able to learn the game by playing it.
  • Fragmented: Performance must be broken into many little games, where winning one game gives little or no direct advantage in future games. Thus consistent wins allow strong inferences on underlying ability.
  • Isolated: Players can’t easily get help from hidden allies outside the game.
  • Status: Chess is seen as very high status, because so many high status people have treated it as high status for so long. Somehow this new game needs to have a shot at achieving a status that high.

If these criteria could be met, high capability people might try to achieve status by consistently winning at this game, the opinions they generate on relevant topics might be more honest and accurate, and the rest of us might then be more inclined to listen to those accurate and relevant opinions.

GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • Ronfar

    Fantasy stock picking?

    • https://angel.co/bmath bmathes

      A good game would have the least amount of Fooled By Randomness (i.e. survivorship bias). Stock picking, arguably, as some of the most.

  • http://badoutcomes.blogspot.com/ Robert H.

    “Let’s design a game that will show which high status people have the best grasp of truth, then reward those people” is such an obvious trick to meet a Culture agent.

  • arch1

    I like the proposal. There would still be the question of whether a person *capable* of honesty and avoiding self delusion, is actually *using* that capability when discussing a given topic.

    Unless you’re proposing the “discussing” of a given topic actually somehow be part of the game itself.

  • Neuronaut

    How about a game with simulated societies, where winning corresponds to maximizing some objective function corresponding to the well-being of the citizens of those societies (or making the most accurate conditional predictions about it). In each simulation, the rules governing the simulation would be different. Then what is rewarded is the ability to infer the rules and bet accordingly. I’m sure this violates all the game design principles, but on the other hand it sounds like a kind of SimCity clone.

  • Romeo Stevens

    Add ELO rankings to TED talks.

  • David Condon

    I would suggest the game should be turn-based, as real-time games reward youth over experience, and so we would end up with a lot of young people who don’t understand anything about the world except this game they’ve played for 8 hours a day since they were 10. The game should also be sufficiently complex. A rock-paper-scissors expert probably wouldn’t provide much of use to the world; at least not due to their experiences with rock-paper-scissors. The game should have time limits, so as not to bore its audience. I can think of a few popular or well known existing examples that spring to mind as possibilities: fantasy football, poker, Go, and Hearthstone. Wikipedia helpfully supplies a list: https://en.wikipedia.org/wiki/List_of_world_championships_in_mind_sports although that list may not be complete as it doesn’t list Hearthstone which is the second highest paying card game in the world after Poker.

    Fantasy Football – This game may be too random. Although there is certainly a lot of skill involved, a lot can change due to injuries or due to unusually good years from certain players, and then playoffs even further increase the amount of randomness. I’m not sure it’s possible for anyone to ever become a world champion at fantasy football.

    Chess – This appears to be more and more a game of memorization. Players memorize a long move set of around 20 moves for their opening, and then learn a bunch of algorithms for optimal play in the end game which is also mostly solved. The only part for any real innovation is in the mid-game. So this should be chosen if our main goal is to identify people who are very good at studying and memorizing complex problems.

    Poker – This is much more algorithm focused. Players have to keep track of how often they play certain moves as well as how often their opponents play certain moves in similar situations. Poker players might be very good at identifying the probabilities involved in situations with incomplete information. However, poker players are working from a relatively simple set of potential combinations. 13 card values and 4 card suits per card, and the card suit is only relevant in a limited number of circumstances. Most combinations can be dismissed as irrelevant. There are maybe less than a thousand relevant possible combinations in a game (not taking into account the betting aspect which is more complicated).

    Go – Here the possibilities are very open ended. I’m not entirely clear on how a Go expert becomes a Go expert, so I’m less certain what Go players would be good at.

    Hearthstone – Like poker, Hearthstone players have to work from incomplete information, but the number of combinations are larger in terms of possibilities created by the cards, but also smaller in that there is no betting involved. More importantly, Hearthstone regularly changes the game by releasing new decks every few months. This allows less room for practiced play of a few combinations to develop, and so Hearthstone players must learn more about the general principles governing the game, and how to adapt to those principles. Hearthstone players must be able quick to quickly change their playing style, and learn a whole new one, so they may be better at considering multiple viewpoints, but they may not be as good at memorizing vast sums of information.

  • Zvi Mowshowitz

    The most fundamental and interesting of these constraints, and the one we should tackle first, is Relevant. The hard one is Status, but I think that (at this stage) is more about making the game insanely great than anything else, and perhaps avoiding low status elements. The things that make a game high status I think are mostly things you want in a game anyway. Fair, Fragmented and Isolated are worth keeping in mind but you do get them by default with most game types.

    So the question we must ask is, what are these relevant topics that we want people to develop more understanding of? What is our theme? Having a unique goal in mind might inspire us to create something new, and can be the restriction that breeds creativity. So… suggestions?

    • http://overcomingbias.com RobinHanson

      How about using something like Amazon Mechanical Turk to invoke one or two random people in the world, and the game is to achieve some outcome with them?

      • Frederic Bush

        Using random people as your gameboard, so to speak, has a number of issues. I think you would want more than one competitor assigned to the same random person, because some people might be more receptive than others. You would also need to anonymize the competitors, and restrict their interactions in some way, to prevent bias on the part of the random person from affecting the game — even if they can’t see the competitors they are interacting with, if there is any sort of openended conversation then competitors who don’t speak fluent unaccented english would be at a big disadvantage in a pool of american people, say.

      • Zvi Mowshowitz

        Interesting idea. Certainly it is a useful skill to be able to get others to do something useful and give them good instructions. I am not as concerned as Frederic about the ‘unfairness’ of who you get, because both that can be one of the useful skills we train and because we could develop ratings for the ‘executors’. We could even have two types of competition, with both ‘instructor’ and ‘executor’ being roles people can play and excel at – you form a team and have to work together.

    • Malte

      I guess that the biggest difference between chess and the “real World” is that chess is a game of perfect information while in the world we have to act with imperfect information.
      I consider the idea of bidding chess (https://arxiv.org/abs/0903.2995) as an good way to improve Relevant,while keeping the others roughly on the same level.

  • Chris Hibbert

    Robin, I was sure the intro was a lead-up to a discussion of Prediction Markets. In what ways to prediction markets fail to meet your criteria?

    At Foresight Exchange, for a long time, we had a competition going. It seems to clearly meet your desiderata for Relevant, Fair, Fragmented, and Isolated. Status isn’t something that you can design in, but having a leaderboard is a way of at least displaying how everyone is doing.

    The weakest attribute, I think, is that there’s a benefit to having done better in the past–that you have a bigger bankroll to play more in the future. But I don’t think that’s a huge barrier; anyone who doesn’t lose all their money can get better over time, and start accumulating wealth that will allow them to continue to participate.

    The fact that the subjects being discussed cover all the socially relevant and impactful topics that people care about seems key to leveraging status.

    • http://overcomingbias.com RobinHanson

      Many topics take a long time to resolve, conflicting with fragmented criteria. And others can give you private tips, conflicting with isolated.

  • Pingback: Rational Feed – deluks917

  • qznc

    I believe Poker is the better chess.

    Although, the popular Texas Holdem variant is a little bit too random. It needs sooo many of the little games. Other variants are more intellectual. Maybe even some games not called Poker, like Pandante.

  • blogospheroid

    Also mentioned by Robert H, but the planet that the protagonist visits in Ian Banks’s “the player of games” had such a game, Azad, where the best players went up the real world hierarchy. It was a starcarft kind of fleet game from what I remember.

    • blogospheroid

      Ok, after reading some wikis, I realise it was a lot more complicated than my simple memories of it.

      • oldoddjobs

        I thought of Player of Games too, heh

  • http://juridicalcoherence.blogspot.com/ Stephen Diamond

    Which raises the question: can we design a game that promotes even more useful honestly? [italics mine]

    Outside of chess, how exactly is honesty of the sort required of chess players useful?

    • Curt Adams

      I think honesty *to yourself* (or resistance to self-delusion) is pretty useful. That’s what chess requires (if you want to be really good).

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        I question whether intellectual honesty acquired in one endeavor generalizes. Does becoming a ruthlessly self-critical chess player make it less likely that one will be an intellectual fraud. Stalin was reportedly a good chess player. (Perhaps not that good, but why not in principle?)

        Put another way, I don’t think a great chess players acquire the self-image of being an honest intellectual. Honesty is instrumental to the chess master’s competitive success, not usually an end in itself.

      • ScottH3

        In short, yes. I think it’s the rule that all principles that lead to success generalize — even to those simply observing. For the actual practitioner it’s a no-brainer.

        At least that’s been my experience.

        And finally, we know Stalin was ruthlessly evil, but do we know he was a fraud in the sense we are talking about here? Stalin’s overall performance as Dictator says no.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        Stalin intentionally falsified history. (See the work of the Dewey Commission – https://en.wikipedia.org/wiki/Dewey_Commission ) Was he ruthlessly self-critical in the manufacture of lies? Quite likely. But this doesn’t keep him from being an intellectual fraud.

        The thesis being considered is that instrumental self-criticism promotes intellectual honesty. We have two issues. The first is how far cognitive habits generalize. But it now seems to me the more important point concerns the very nature of intellectual honesty, which I take to be part of an identity rather than a collection of instrumental habits. The nature of scientific integrity was described by Feynman in “Cargo Cult Science.” (http://calteches.library.caltech.edu/51/2/CargoCult.htm )

  • http://www.sanger.dk Pepper

    The stock market could be seen as a big prediction tournament.

  • DuncanHaywood

    Off the top of my head something like MUN or Mafia would accomplish a usefulness in social interactions when approached within a community and with the principles of competition laid out above.

  • Jacek Stopa

    Chess players of a high caliber will be extremely cautious about not making mistakes; usually the higher your chess level, the more you hate losing, and the more afraid you are to make certain types of mistakes. You will develop a strong desire for a feeling of completeness in your strategies and, over years, this will translate into other domains of your life. That’s why you see many chessplayers sticking to limited, but robust, opening repertoires instead of striving for adventures, as their experience has taught them that knowing one’s turf well is much more important than trying to get some last-minute excitement of choosing a dubious, but more interesting, line (they have been punished often enough). In a sense, then, each game played and properly analyzed will make you more honest with yourself. The more such games played, the higher the rate of becoming more self-aware, or honest with yourself. Now, not every chessplayer at a level comparable with Kasparov’s exhibits so much interest in so many areas outside of chess as he does. Actually, it’s pretty safe to say, that relatively few do, with the rest just having fully committed to excelling at the game and only at the game. Myself, as a chess grandmaster, although not nearly at the top level, I value my chess experience greatly and clearly see how it gives me steam to climb other learning curves which have nothing to do with the game. The feeling that I may be making a mistake somewhere is essentially always there.
    As far as designing another game with the intention of increased rate of becoming more honest, I doubt such effort would yield any substantial improvement over chess, as for it to succeed it would need to be the precision of the game rules to be the contributing factor. And to my taste, and 20 years of experience of playing/ coaching chess, it’s enough for the game to be complex enough to secure a life-long learning experience to humble the player and hence induce the feeling of weakness-awareness, or honesty. Even the current World Chess Champion and the strongest player of all times, Magnus Carlsen, likes to express his disappointment with his own level of play, and the truth is that he cannot realistically compete with computer programs. If Carlsen gets repeatedly humbled about his play, I simply cannot see along which trajectory a more honesty-inducing game could be constructed.

  • Southron

    Ah yes, the usual literature on game design. Which of course all of us know. However, just in case some confiding person, perhaps very young, were to glance through this discussion some time, perhaps we might refer to a roundup

    • James G

      I’d second a roundup of game design literature. Sounds fascinating!

  • David Levine

    Phil Tetlock and colleagues have run long forecasting tournaments that show how to detect, reward and teach reducing overconfidence, learning from others, and being willlng to update. Those skills are key elements of this challenge’s “brutal honesty.” Turning their approach into a more playable game is a challenge. Nevertheless, I suspect any winner of Hanson’s challenge can learn a lot from Tetlock’s approach.