| Stuff |
[May. 22nd, 2010|12:51 am]
Scott
|
B. F. Skinner, who has been mentioned on this blog before in the context of his not locking his young daughter in a box for seven years, is possibly best known for his studies on reinforcement learning. He locked hundreds of pigeons (but not his young daughter) in special contraptions called "Skinner boxes", inside of which were various levers and a food distribution system. Skinner spent several years giving the birds food for pulling levers until he knew everything there was to know about lever-pressing pigeons.
Skinner found certain patterns of reinforcement made the pigeons press more levers than other patterns. What he called "ratio reinforcement" - giving food according to number of lever presses" - made the pigeons press the lever more often than "interval reinforcement" - giving food on the first lever press after a certain amount of time. And what he called "variable schedules" - giving food randomly on average every ten lever-presses - made the pigeons press the lever more often than "fixed reinforcement" - giving food exactly every ten lever-presses.
That makes the most powerful reinforcement schedule "variable ratio" - giving food randomly after a certain number of lever presses. This reinforcement schedule is one hundred percent evil (though maybe not lock-your-daughter-in-a-box evil) and people can and have used it to destroy lives.
The textbook example of variable ratio reinforcement is a slot machine. You give it money, and at random after a certain number of plays it gives you a reward, getting people addicted and desperate for another push. You can bet casino designers have read Skinner - heck, they even kept the lever mechanism! And David Wong writes that now online game designers have are reading Skinner too. This is part of the reason games like World of Warcraft and Farmville are so addictive (also, in the Farmville case, the producer sold his soul to Satan).
Only slightly less scary than variable ratio reinforcement is variable interval reinforcement. And I recently noticed that practically everything I do on the Internet is a variable interval reinforcement and that I've become completely addicted to them.
Every day, I check Harry Potter and the Methods of Rationality to see if a new chapter's been published. Which could be worse...except that it's actually like ten times a day, just clicking and reloading, realizing that one of these times I'm going to get lucky.
Every day, I check OKCupid to see whether a beautiful and fascinating girl with non-cookie-cutter interests has, for some inexplicable reason, decided to move to southwest Ireland and is looking for romance. Once or twice a year, it turns out someone has (although it's never quite worked out). And that reinforcement - variable interval schedule, note - keeps me going back to the site several times a day, just to make sure.
That's not even counting all the times I reload LiveJournal or Less Wrong or Shireroth to see if there have been any new interesting posts or anyone's upvoted me or commented on my stuff. Same principle, same hours a day of pointless clicking.
Really, I could save myself a lot of time by just checking all of them once per day, or even less frequently. That next chapter of Harry Potter, that beautiful girl who's always wanted to move to southwest Ireland to take a job in its highly-competitive and dynamic sheep-counting industry - they're not going away. But the thought of limiting my clicking behavior fills me with a sense of dread and total mental resistance.
I wonder if that's how the pigeons felt about limiting their pecks on those levers. |
|