Von Neumann and Realpolitik

"Right, as the world goes, is only in question between equals in power, while the strong do what they can and the weak suffer what they must." -- Thucydides, Melian Dialogue.

 

Von Neumann, Feynman, and Ulam.

Adventures of a Mathematician (Ulam): ... Once at Christmas time in 1937, we drove from Princeton to Duke University to a meeting of the American Mathematical Society. ...

As we passed the battlefields of the Civil War, Johnny recounted the smallest details of the battles. His knowledge of history was really encyclopedic, but what he liked and knew best was ancient history. He was a great admirer of the concise and wonderful way the Greek historians wrote. His knowledge of Greek enabled him to read Thucydides, Herodotus, and others in the original; his knowledge of Latin was even better.

The story of the Athenian expedition to the island of Melos, the atrocities and killings that followed, and the lengthy debates between the opposing parties fascinated him for reasons which I never quite understood. He seemed to take a perverse pleasure in the brutality of a civilized people like the ancient Greeks. For him, I think it threw a certain not-too-complimentary light on human nature in general. Perhaps he thought it illustrated the fact that once embarked on a certain course, it is fated that ambition and pride will prevent a people from swerving from a chosen course and inexorably it may lead to awful ends, as in the Greek tragedies. Needless to say this prophetically anticipated the vaster and more terrible madness of the Nazis. Johnny was very much aware of the worsening political situation. In a Pythian manner, he foresaw the coming catastrophe. ...

... I will never forget the scene a few days before he died. I was reading to him in Greek, from his worn copy of Thucydides, a story he liked especially about the Athenians' attack on Melos, and also the speech of Pericles. He remembered enough to correct an occasional mistake or mispronunciation on my part.

... there was nothing small about his interests ... He was unique in this respect. Unique, too, were his overall intelligence, breadth of interest, and absolute feeling for the difference between the momentary technical work and the great lines of the life of the mathematical tree itself and its role in human thought.

Michael Anton: Inside the Trump White House

Michael Anton is head of strategic communications for the National Security Council. See related Politico article.

The Atlantic: Michael Anton warned last year that 2016 was the Flight 93 election: “Charge the cockpit or you die.”

Americans charged. Donald Trump became president of the United States. And Anton, the author of that now-notorious essay, is helping to fly the plane—running communications for the National Security Council.

Anton cuts a curious figure through the Trump White House. A thoroughly educated dandy, his writings are at the core of an effort to construct an intellectual framework around the movement that elected a president who has shown no inclination to read books and who speaks in an unpretentious New York vernacular. ...

Yann LeCun on Unsupervised Learning

This is a recent Yann LeCun talk at CMU. Toward the end he discusses recent breakthroughs using GANs (Generative Adversarial Networks, see also Ian Goodfellow here and here).

LeCun tells an anecdote about the discovery of backpropagation. The first implementation of the algorithm didn't work, probably because of a bug in the program. But they convinced themselves that the reason for the failure was that the network could easily get caught in a local minimum which prevents further improvement. It turns out that this is very improbable in high dimensional spaces, which is part of the reason behind the great success of deep learning. As I wrote here:

In the limit of high dimensionality a critical point is overwhelmingly likely to be a saddlepoint (have at least one negative eigenvalue). This means that even though the surface is not strictly convex the optimization is tractable.

This (free version!) new textbook on deep learning by Goodfellow, Bengio, and Courville looks very good. See also Michael Nielsen's book.

If I were a young person I would be working in this area (perhaps with an eye toward applications in genomics, or perhaps working directly on the big problem of AGI). I hope after I retire they will let me hang out at one of the good AI places like Google Brain or Deep Mind :-)

History of Bayesian Neural Networks

This talk gives the history of neural networks in the framework of Bayesian inference. Deep learning is (so far) quite empirical in nature: things work, but we lack a good theoretical framework for understanding why or even how. The Bayesian approach offers some progress in these directions, and also toward quantifying prediction uncertainty.

I was sad to learn from this talk that David Mackay passed last year, from cancer. I recommended his book Information theory, inference and learning algorithms back in 2007.

Yarin Gal's dissertation Uncertainty in Deep Learning, mentioned in the talk.

I suppose I can thank my Caltech education for a quasi-subconscious understanding of neural nets despite never having worked on them. They were in the air when I was on campus, due to the presence of John Hopfield (he co-founded the Computation and Neural Systems PhD program at Caltech in 1986). See also Hopfield on physics and biology.

Amusingly, I discovered this talk via deep learning: YouTube's recommendation engine, powered by deep neural nets, suggested it to me this Saturday afternoon :-)

The Rise and Fall of the Meritocracy (BBC podcast)

British socialist Michael Young coined the term meritocracy in his 1958 dystopian satire The Rise of the Meritocracy. He realized even then that a system which rewarded individuals fairly, based on ability and effort, would likely lead to genetic class stratification, due to the heritability of traits. His son Toby, a journalist for the Spectator, explores this topic in an excellent BBC podcast, featuring researchers such as Robert Plomin and me.

In the future, will we redistribute genetic wealth as well as material wealth?

Toby Young's six problems with meritocracy

The Rise of the Meritocracy is a dystopian satire written almost sixty years ago by pioneering sociologist Michael Young. It imagined a modern society uncannily like our own and coined the term meritocracy.

Michael Young's son Toby, a journalist for the Spectator, has been asking if his asks if his father's dark prophesy is correct. Here are Toby Young's six problems with a meritocracy.

1. Where meritocracy came from

The word ‘meritocracy’ was coined by my father, a left-wing sociologist called Michael Young, to describe a dystopian society of the future. In his 1958 book The Rise of the Meritocracy, he imagines a 21st Century Britain in which status is determined by a combination of IQ and effort. He acknowledged that this was fairer than an aristocratic society in which status is simply passed on from parents to their children, but it was precisely because meritocracy gave a patina of legitimacy to the inequalities thrown up by free market capitalism that he disapproved of it.

2. Is a meritocratic society fairer?

The political philosopher John Rawls pointed out that a meritocratic society isn’t necessarily fairer than an aristocratic one. After all, the qualities that meritocracy rewards – exceptional intelligence and drive – are, for the most part, natural gifts that people are born with. Since successful people have done nothing to deserve those talents, they don’t deserve the rewards they bring any more than they deserve to inherit a fortune.

3. Complete equality of opportunity

For a society to be 100% meritocratic, you need complete equality of opportunity. But the only way to guarantee that is to remove children from their parents at birth and raise them in identical circumstances. If you don’t do that, the socio-economic status of a child’s parents will inevitably affect that child’s life chances.

4. Is it in the genetics?

According to the political scientist Charles Murray, meritocracy inevitably leads to a genetically-based caste system. Why? Because the traits selected for by the meritocratic sorting principle are genetically-based and, as such, likely to be passed on from parents to their children. Genetic variation means some highly able children will be born to people of average and below average intelligence, but the children of the meritocratic elite will, in aggregate, always have a competitive advantage and over several generations that leads to social ossification.

5. Noblesse oblige

One of the things my father disliked about meritocracy was that it engendered a sense of entitlement amongst the most successful. Because they regard their elevated status as thoroughly deserved, they’re not burdened by a sense of noblesse oblige. At least in an aristocratic society, members of the lucky sperm club are afflicted by guilt and self-doubt and, as such, tend to be a bit nicer to those below them.

6. Is America the most meritocratic country?

Americans like to tell themselves that they live in the most meritocratic country in the world but, in fact, it may be one of the least. In most international league tables of inter-generational social mobility, which measure the chances a child born into one class has of moving into another over the course of their lifetime, America is at the bottom.

[ Of course, lack of mobility could also result in a society that is both meritocratic and already somewhat stratified by genetics. See Income, Wealth, and IQ and US economic mobility data. ]

I wonder whether even Michael Young was aware that the British implementation of meritocracy through competitive civil service and university entrance examinations (a century before his book The Rise of the Meritocracy) was a deliberate adoption of Chinese ways.

Les Grandes Ecoles Chinoises: ... the British and French based their civil service and educational examination systems on the much older Chinese model ...

... French education was really based on the Chinese system of competitive literary examinations, and ... the idea of a civil service recruited by competitive examinations undoubtedly owed its origins to the Chinese system which was popularized in France by the philosophers, especially Voltaire. ...

Summary of the case of Britain and colonial India can be found here. Amusingly, 19th century British writers opposed to the new system of exams referred to it as "... an adopted Chinese culture" (p. 304-305).

Penalized regression from summary statistics

One of the difficulties in genomics is that when DNA donors are consented for a study, the agreements generally do not allow sharing (aggregation) of genomic data across multiple studies. This leads to isolated silos of data that can't be fully shared. However, computations can be performed on one silo at a time, with the results ("summary statistics") shared within a larger collaboration. Most of the leading GWAS collaborations (e.g., GIANT for height, SSGAC for cognitive ability) rely on shared statistics. Simple regression analysis (one SNP at a time) can be conducted using just summary statistics, but more sophisticated algorithms cannot. These more sophisticated methods can generate a better phenotype predictor, using less data, than a SNP by SNP analysis.

For example, the objective function used in LASSO (L1-penalized regression) is of the form

where, for the genomics problem, y is the phenotype vector, X the matrix of genomes, beta the vector of effect sizes, and lambda the penalization. Optimization of this function seems to require access to the full matrix X and vector y -- i.e., requires access to potentially all the genomes and phenotypes at once. Is there a modified version of the algorithm that works on summary statistics, where only subsets of X and y are available? Carson Chow has advocated this approach to me for some time. If one can separately estimate X'X (LD matrix of genomic correlations), and gather X'y (phenotype-SNP correlations) from summary statistics, then LASSO over silo-ed data may become a reality. Of course, the devil is in the details. The paper below describes an approach to this problem.

Polygenic scores via penalized regression on summary statistics

Timothy Mak, Robert Milan Porsch, Shing Wan Choi, Xueya Zhou, Pak Chung Sham
doi: https://doi.org/10.1101/058214

Polygenic scores (PGS) summarize the genetic contribution of a person's genotype to a disease or phenotype. They can be used to group participants into different risk categories for diseases, and are also used as covariates in epidemiological analyses. A number of possible ways of calculating polygenic scores have been proposed, and recently there is much interest in methods that incorporate information available in published summary statistics. As there is no inherent information on linkage disequilibrium (LD) in summary statistics, a pertinent question is how we can make use of LD information available elsewhere to supplement such analyses. To answer this question we propose a method for constructing PGS using summary statistics and a reference panel in a penalized regression framework, which we call lassosum. We also propose a general method for choosing the value of the tuning parameter in the absence of validation data. In our simulations, we showed that pseudovalidation often resulted in prediction accuracy that is comparable to using a dataset with validation phenotype and was clearly superior to the conservative option of setting the tuning parameter of lassosum to its lowest value. We also showed that lassosum achieved better prediction accuracy than simple clumping and p-value thresholding in almost all scenarios. It was also substantially faster and more accurate than the recently proposed LDpred.

See also Bayesian large-scale multiple regression with summary statistics from genome-wide association studies.

National Geographic: How Humans Are Shaping Our Own Evolution

See also A Brief History of the Future, as told to the Masters of the Universe and Super-Intelligent Humans are Coming.

National Geographic: How Humans Are Shaping Our Own Evolution:

... Unlike our forebears, we may soon not need to wait for evolution to fix the problem. In 2013 Nick Bostrom and Carl Shulman, two researchers at the Future of Humanity Institute, at Oxford University, set out to investigate the social impact of enhancing intelligence, in a paper for Global Policy. They focused on embryo selection via in vitro fertilization. With IVF, parents can choose which embryo to implant. By their calculations, choosing the “most intelligent embryo” out of any given 10 would increase a baby’s IQ roughly 11.5 points above chance. If a woman were willing to undergo more intensive hormone treatments to produce eggs faster—“expensive and burdensome,” as the study notes with understatement—the value could grow.

The real benefit, though, would be in the compound gain to the recipient’s descendants: After 10 generations, according to Shulman, a descendant might enjoy an IQ as much as 115 points higher than his or her great-great-great-great-great-great-great-great-grandmother’s. As he pointed out to me, such a benefit is built on extremely optimistic assumptions, but at the least the average recipient of this genetic massaging would have the intelligence equal to a genius today. Using embryonic stem cells, which could be converted into sperm or ova in just six months, the paper notes, might yield far faster results. Who wants to wait two centuries to be the scion of a race of geniuses? Shulman also mentioned that the paper omitted one obvious fact: “In 10 generations there will likely be computer programs that outperform even the most enhanced human across the board.”

There’s a more immediate objection to this scenario, though: We don’t yet know enough about the genetic basis for intelligence to select for it. One embryo doesn’t do advanced calculus while another is stuck on whole numbers. Acknowledging the problem, the authors claim that the ability to select for “modest cognitive enhancement” may be only five to 10 years off.

At first glance this would seem improbable. The genetic basis of intelligence is very complex. Intelligence has multiple components, and even individual aspects—computational ability, spatial awareness, analytic reasoning, not to mention empathy—are clearly multigenetic, and all are influenced by environmental factors as well. Stephen Hsu, vice president for research at Michigan State University, who co-founded the Cognitive Genomics Lab at BGI (formerly Beijing Genomics Institute), estimated in a 2014 article that there are roughly 10,000 genetic variants likely to have an influence on intelligence. That may seem intimidating, but he sees the ability to handle that many variants as nearly here—“in the next 10 years,” he writes—and others don’t think you’d need to know all the genes involved to start selecting smarter embryos. “The question isn’t how much we know or don’t know,” Church says. “It’s how much we need to know to make an impact. ..."

On the genetic architecture of intelligence and other quantitative traits
https://arxiv.org/abs/1408.3421

Somewhere ... It's happening...

Chomsky: Russia conspiracy theories "a joke"

Chomsky on the current media / left obsession with anti-Russia conspiracy theories. I guess he and Trump are both Putin puppets... Oops, except Trump just attacked Assad and risked killing Russian soldiers. So that just leaves Chomsky.

NOAM CHOMSKY: It’s a pretty remarkable fact that—first of all, it is a joke. Half the world is cracking up in laughter. The United States doesn’t just interfere in elections. It overthrows governments it doesn’t like, institutes military dictatorships. Simply in the case of Russia alone—it’s the least of it—the U.S. government, under Clinton, intervened quite blatantly and openly, then tried to conceal it, to get their man Yeltsin in, in all sorts of ways. So, this, as I say, it’s considered—it’s turning the United States, again, into a laughingstock in the world.

So why are the Democrats focusing on this? In fact, why are they focusing so much attention on the one element of Trump’s programs which is fairly reasonable, the one ray of light in this gloom: trying to reduce tensions with Russia? That’s—the tensions on the Russian border are extremely serious. They could escalate to a major terminal war. Efforts to try to reduce them should be welcomed. ...

So, meanwhile, this one topic is the primary locus of concern and critique, while, meanwhile, the policies are proceeding step by step, which are extremely destructive and harmful. So, you know, yeah, maybe the Russians tried to interfere in the election. That’s not a major issue. Maybe the people in the Trump campaign were talking to the Russians. Well, OK, not a major point, certainly less than is being done constantly. And it is a kind of a paradox, I think, that the one issue that seems to inflame the Democratic opposition is the one thing that has some justification and reasonable aspects to it.

... Well, you can understand why the Democratic Party managers want to try to find some blame for the fact—for the way they utterly mishandled the election and blew a perfect opportunity to win, handed it over to the opposition

... NATO maneuvers are taking place hundreds of yards from the Russian border. The Russian jet planes are buzzing American planes. This—something could get out of hand very easily. ... people like William Perry, who has a distinguished career and is a nuclear strategist and is no alarmist at all, is saying that we’re back to the—this is one of the worst moments of the Cold War, if not worse.

See also Trump: Give Peace a Chance, Obama: "Don't do stupid sh*t", and Trump, Putin, Stephen Cohen, Brawndo, and Electrolytes.

Sex Differences In The Adult Human Brain: UK Biobank data

Male brains exhibit larger variance across all morphological measures. (Don't tell Larry Summers! ;-)

Note, as far as I can tell the authors don't normalize the SD by mean value for each gender to obtain the typical percentage fluctuation (a dimensionless quantity). The male brain is about 10% larger and each of the subregions is roughly that much bigger as well. If you divide the larger male SD by the larger male mean for each morphology the effect is much smaller than tabulated in the second figure below.

Sex Differences In The Adult Human Brain: Evidence From 5,216 UK Biobank Participants

doi: https://doi.org/10.1101/123729

Sex differences in human brain structure and function are of substantial scientific interest because of sex-differential susceptibility to psychiatric disorders and because of the potential to explain sex differences in psychological traits. Males are known to have larger brain volumes, though the patterns of differences across brain subregions have typically only been examined in small, inconsistent studies. In addition, despite common findings of greater male variability in traits like intelligence, personality, and physical performance, variance differences in the brain have received little attention. Here we report the largest single-sample study of structural and functional sex differences in the human brain to date (2,750 female and 2,466 male participants aged 44-77 years). Males had higher cortical and sub-cortical volumes, cortical surface areas, and white matter diffusion directionality; females had thicker cortices and higher white matter tract complexity. Considerable overlap between the distributions for males and females was common, and subregional differences were smaller after accounting for global differences. There was generally greater male variance across structural measures. The modestly higher male score on two cognitive tests was partly mediated via structural differences. Functional connectome organization showed stronger connectivity for males in unimodal sensorimotor cortices, and stronger connectivity for females in the default mode network. This large-scale characterisation of neurobiological sex differences provides a foundation for attempts to understand the causes of sex differences in brain structure and function, and their associated psychological and psychiatric consequences.

Susan Rice and U.S. person information "derived solely from raw SIGINT"

I hope this scandal will focus additional attention on massive bulk collection and preservation of private communications of US citizens by NSA.

Media discussion continues to focus on "unmasking" = dissemination of identities of US individuals. However, I have yet to see discussion of whether someone like Rice could order specific database searches (e.g., by NSA, of preserved records) on a specific individual to acquire intercepts such as voice transcripts, emails, etc. It doesn't seem to become an unmasking until that information is distributed in the form of an intelligence report (or, is such a request automatically an unmasking?). The search results alone constitute an invasion of individual privacy. It is unclear to me who has access to such results, and under what conditions the searches can be requested. There are well known instances of NSA employees abusing these powers: see LOVEINT. Could the White House order something similar without a record trail? (See excerpt added below.)

Bloomberg: Susan Rice Sought Names in Trump Intel, Says Eli Lake

Former national security adviser Susan Rice made multiple requests for the identities of people connected to the transition team of Donald Trump contained in raw intelligence reports, according to U.S. officials familiar with the matter. Bloomberg View columnist Eli Lake has the details.

From Nunes, Trump, Obama and Who Watches the Watchers?, this is the legal standard that the Susan Rice unmaskings will be judged by:

Section VI: ... An IC element may disseminate U.S. person information "derived solely from raw SIGINT" under these procedures ... if ... the information is “necessary to understand the foreign intelligence or counterintelligence information,”

Richard Haas notes that this kind of activity on the part of Susan Rice and NSC staff is only justifiable under "extraordinary circumstances"!



Added (from comments):

The Observer: ... In addition, Rice didn’t like to play by the rules, including the top-secret ones. On multiple occasions, she asked the NSA to do things they regarded as unethical and perhaps illegal. When she was turned down — the NSA fears breaking laws for any White House, since they know they will be left holding the bag in the end — Rice kept pushing.

As a longtime NSA official who experienced Rice’s wrath more than once told me, “We tried to tell her to pound sand on some things, but it wasn’t allowed—we were always overruled.” On multiple occasions, Rice got top Agency leadership to approve things which NSA personnel on the front end of the spy business refused. This means there may be something Congress and the FBI need to investigate here.

...

John Schindler is a security expert and former National Security Agency analyst and counterintelligence officer. A specialist in espionage and terrorism, he’s also been a Navy officer and a War College professor. He’s published four books and is on Twitter at @20committee.

Dialog 2017

Too busy to take many photos, but here are a few. First one is of a conversation between Tyler Cowen and Peter Thiel.

Dialog is an biannual 2-day thought retreat, gathering 150 global leaders to discuss how to change the world. Dialog was created in 2006 to bring together global leaders across industries to discuss some of the most pressing world issues, to provide an opportunity for cross-pollination & collaboration.

There are no speakers. No panels. All attendees participate in break-out facilitated discussions. And we limit the discussion to only 150 global leaders who can have an impact now and emerging leaders who can help implement the plans we develop.

There are no speeches, just many coordinated, moderated break-out discussions of 6-15 people. The agenda is determined by the attendees directly, based on their interests and needs.

Dialog is an invite-only retreat and we carefully curate all participants. Dialog is 100% off-the-record and not-for-attribution. Dialog is hosted by Auren Hoffman and Peter Thiel.

The brute tyranny of g-loading: Lawrence Krauss and Joe Rogan

I love Joe Rogan -- he has an open, inquisitive mind and is a sharp observer of life and human society. See, for example, this interview with Dan Bilzerian about special forces, professional poker, sex, drugs, heart attacks, life, happiness, hedonic treadmill, social media, girls, fame, prostitution, money, steroids, stem cell therapy, and plenty more.

I know Lawrence Krauss quite well -- he and I work in the same area of theoretical physics. However, the 20+ minute opening segment in which Krauss tries to explain gauge symmetry (1, 2, 3) to Joe is downright painful. Some things are just conceptually hard, and are built up from other concepts that are themselves non-obvious.

Gauge symmetry is indeed central to modern theoretical physics -- all of the known forces of nature are gauge interactions. I've been at an uncountable number of cocktail parties (sometimes with other professors) where I've tried to explain this concept to someone as sincerely interested as Rogan is in the video. Who doesn't like to hear about fundamental laws of Nature and deep principles of physical reality?

No matter how clearly a very g-loaded concept is explained, it is challenging for the typical person to comprehend. (This is almost a definition.) Many ideas in physics are challenging even to college professors. One sad aspect of the Internet is that there isn't any significant discussion forum or blog comment section where even much simpler concepts such as regression to the mean are understood by all the participants.

Listening to the conversation between Joe and Lawrence about gauge theory and the Higgs field, I couldn't help but think of this Far Side cartoon:

Oppenheimer: Mathematics is "an immense enlargement of language, an ability to talk about things which in words would be simply inaccessible."

See also this Reddit discussion of the podcast episode.

Just Like Heaven

Musical intermission. Choose your favorite!

For me it's not the same without that 80's synth...

Robots Proctor Online Exams

For background on this subject, see How to beat online exam proctoring. It is easy for clever students to beat existing security systems for online exams. Enterprising students could even set up "cheating rooms" that make it easy for test takers to cheat. Judging by the amount of traffic this old post gets, cheating on online exams is a serious problem.

Machine learning to the rescue! :-) The machines don't have to be 100% accurate in detection -- they can merely flag suspicious moments in the data and ask a human proctor to look more carefully. This makes the overall system much more scalable.

The monitoring data (e.g., video from webcam + pov cam) from a particular exam could potentially be stored forever. In an extreme case, a potential employer who wants to be sure that Johnny passed the Python coding (or psychometric g) exam for real could be granted access to the stored data by Johnny, to see for themselves.

Automated Online Exam Proctoring
Atoum, Chen, Liu, Hsu, and Liu
IEEE Transactions on Multimedia

Abstract:
Massive open online courses (MOOCs) and other forms of remote education continue to increase in popularity and reach. The ability to efficiently proctor remote online examinations is an important limiting factor to the scalability of this next stage in education. Presently, human proctoring is the most common approach of evaluation, by either requiring the test taker to visit an examination center, or by monitoring them visually and acoustically during exams via a webcam. However, such methods are labor-intensive and costly. In this paper, we present a multimedia analytics system that performs automatic online exam proctoring. The system hardware includes one webcam, one wearcam, and a microphone, for the purpose of monitoring the visual and acoustic environment of the testing location. The system includes six basic components that continuously estimate the key behavior cues: user verification, text detection, voice detection, active window detection, gaze estimation and phone detection. By combining the continuous estimation components, and applying a temporal sliding window, we design higher level features to classify whether the test taker is cheating at any moment during the exam. To evaluate our proposed system, we collect multimedia (audio and visual) data from 24 subjects performing various types of cheating while taking online exams. Extensive experimental results demonstrate the accuracy, robustness, and efficiency of our online exam proctoring system.

This work is related to the issued patent

Online examination proctoring system
WO 2014130769 A1

ABSTRACT
The system to proctor an examination includes a first camera (10) worn by the examination taking subject (12) and directed to capture images in subject's field of vision. A second camera (14) is positioned to record an image of the subject's face during the examination. A microphone (26) captures sounds within the room, which are analyzed to detect speech utterances. The computer system (8) is programmed to store captured images from said first camera. The computer (16) is also programmed to issue prompting events instructing the subject to look in a direction specified by the computer at event intervals not disclosed to subject in advance and to index for analysis the captured images in association with indicia corresponding to the prompting events.

Publication number WO2014130769 A1
Publication type Application
Application number PCT/US2014/017584
Publication date Aug 28, 2014
Filing date Feb 21, 2014
Priority date Feb 25, 2013
Also published as US9154748, US20140240507
Inventors Stephen Hsu, Xiaoming Liu, Xiangyang Alexander LIU
Applicant Board Of Trustees Of Michigan State University

The weirdest academic seminar ever

... but in a good way! :-)

Beijing-based economist and rapper Andrew Dougherty ("Big Daddy Dough") presents The Redprint: Rhyme and Reason in the Riddle Kingdom at the BYU Kennedy Center.




This is his classic cover Beijing State of Mind:




Dougherty interview on Sinica podcast.

With Kasparov in NYC

We were together on a panel talking about AI. I challenged him to a quick game of blitz but he declined ;-)  The moderator is Josh Wolfe of Lux Capital.

More Kasparov on this blog.

Nunes, Trump, Obama and Who Watches the Watchers?

I've made this separate entry from the update to my earlier discussion FISA, EO 12333, Bulk Collection, and All That. I believe the Nunes revelations from yesterday support my contention that the Trump team intercepts are largely "incidental" collections (e.g., bulk collections using tapped fiber, etc.) under 12333, and the existence of many (leaked) intel reports featuring these intercepts is likely a consequence of Obama's relaxation of the rules governing access to this bulk data. At least, the large number of possible leakers helps hide the identities of the actual leakers!

EO12333 + Obama OKs unprecedented sharing of this info as he leaves office = recent leaks? Note the use of the term "incidentally" and the wide dissemination (thanks to Obama policy change as he left office).

WSJ: ... “I recently confirmed that on numerous occasions the intelligence community incidentally collected information about U.S. citizens involved in the Trump transition,” Mr. Nunes said, reading a brief statement to reporters on Capitol Hill on Wednesday afternoon. “Details about U.S. persons associated with the incoming administration—details with little or no apparent foreign intelligence value—were widely disseminated in intelligence community reporting.”

... Mr. Nunes added that it was “possible” the president himself had some of his communication intercepted, and has asked the Federal Bureau of Investigation, National Security Agency and other intelligence agencies for more information.

The change put in place as Obama left office is probably behind the large number of circulating reports that feature "incidentally" captured communications of the Trump team. The NYTimes article below is from February.

NYTimes: ... Until now, National Security Agency analysts have filtered the surveillance information for the rest of the government. They search and evaluate the information and pass only the portions of phone calls or email that they decide is pertinent on to colleagues at the Central Intelligence Agency, the Federal Bureau of Investigation and other agencies. And before doing so, the N.S.A. takes steps to mask the names and any irrelevant information about innocent Americans.

[ So FBI is only getting access to this data for the first time. It is interesting that Nunes said that NSA would comply with his request for more information but that FBI has not complied. It seems possible that FBI does not yet have good internal controls over how its agents use these new privileges. ]

The new system would permit analysts at other intelligence agencies to obtain direct access to raw information from the N.S.A.’s surveillance to evaluate for themselves. If they pull out phone calls or email to use for their own agency’s work, they would apply the privacy protections masking innocent Americans’ information — a process known as “minimization” — at that stage, Mr. Litt said.

... FISA covers a narrow band of surveillance: the collection of domestic or international communications from a wire on American soil, leaving most of what the N.S.A. does uncovered. In the absence of statutory regulation, the agency’s other surveillance programs are governed by rules the White House sets under a Reagan-era directive called Executive Order 12333.

... [it is unclear what] rules say about searching the raw data using names or keywords intended to bring up Americans’ phone calls or email that the security agency gathered “incidentally” under the 12333 surveillance programs ...

It appears that the number of individuals allowed to search bulk, incidentally collected data has been enlarged significantly. Who watches these watchers? (There must now be many thousands...)

Sophos: Obama administration signs off on wider data-sharing for NSA ... Patrick Toomey, a lawyer for the American Civil Liberties Union (ACLU), put it in an interview with the New York Times, 17 intelligence agencies are now going to be “rooting… through Americans’ emails with family members, friends and colleagues, all without ever obtaining a warrant”.

The new rules mean that the FBI, the CIA, the DEA, and intelligence agencies of the US military’s branches and more, will be able to search through raw signals intelligence (SIGINT): intercepted signals that include all manner of people’s communications, be it via satellite transmissions, phone calls and emails that cross network switches abroad, as well as messages between people abroad that cross domestic network switches.

Added: Quick and dirty summary of new rules governing access to raw SIGINT. Note, lots of room for abuse in what I quote below:

Section III: ... NSA may make raw SIGINT available through its own systems, through a shared IC or other government capability (like a cloud environment), or by transferring the information to the IC element's information systems.

Section V: ... Communications solely between U.S. persons “inadvertently retrieved during the selection of foreign communications” will be destroyed except if they contain significant foreign intelligence or counterintelligence as determined by the IC element.

Section VI: ... An IC element may disseminate U.S. person information "derived solely from raw SIGINT" under these procedures ... if ... the information is “necessary to understand the foreign intelligence or counterintelligence information,”

Here are the entities who now have access (thanks Obama!) to raw SIGINT, and seem to have the discretionary power to "unmask" US citizens appearing in the data.

IC elements are defined under 3.5(h) of E.O. 12333 as: (1) The Office of the Director of National Intelligence; (2) The Central Intelligence Agency; (3) The National Security Agency; (4) The Defense Intelligence Agency; (5) The National Geospatial-Intelligence Agency; (6) The National Reconnaissance Office; (7) The other offices within the Department of Defense for the collection of specialized national foreign intelligence through reconnaissance programs; (8) The intelligence and counterintelligence elements of the Army, the Navy, the Air Force, and the Marine Corps; (9) The intelligence elements of the Federal Bureau of Investigation; (10) The Office of National Security Intelligence of the Drug Enforcement Administration; (11) The Office of Intelligence and Counterintelligence of the Department of Energy; (12) The Bureau of Intelligence and Research of the Department of State; (13) The Office of Intelligence and Analysis of the Department of the Treasury; (14) The Office of Intelligence and Analysis of the Department of Homeland Security; (15) The intelligence and counterintelligence elements of the Coast Guard; and (16) Such other elements of any department or agency as may be designated by the President, or designated jointly by the Director and the head of the department or agency concerned, as an element of the Intelligence Community.

FISA, EO 12333, Bulk Collection, and All That

Some basic questions for the experts:

1. To what extent does EO12333 allow surveillance of US individuals without FISA warrant?

2. To what extent are US voice conversations recorded via bulk collection (and preserved for, e.g., 5 or more years)? The email answer is clear ... But now automated voice recognition and transcription make storage of voice conversations much more scalable.

3. To what extent do Five Eyes intel collaborators have direct access to preserved data?

4. Are "experts" and media pundits and Senators even asking the right questions on this topic? For example, can stored bulk-collected voice data from a US individual be accessed by NSA without FISA approval by invoking 12333? How can one prevent a search query on stored data from producing results of this type?

See, e.g., Overseas Surveillance in an Interconnected World (Brennan Center for Justice at NYU School of Law), ACLU.org, and Executive Order 12333 (epic.org):

EPIC has tracked the government's reliance on EO 12333, particularly the reliance on Section 1:12(b)(13), which authorizes the NSA to provide "such administrative and technical support activities within and outside the United States as are necessary to perform the functions described in sections (1) through (12) above, including procurement." This provision appears to have opened the door for the NSA's broad and unwarranted surveillance of U.S. and foreign citizens.

Executive Order 12333 was signed by President Ronald Reagan on December 4, 1981. It established broad new surveillance authorities for the intelligence community, outside the scope of public law. EO 12333 has been amended three times. It was amended by EO 13284 on January 23, 2003 and was then amended by EO 13555 on August 27, 2004. EO 13555 was subtitled "Strengthened Management of the Intelligence Community" and reflected the fact that the Director of National Intelligence (DNI) now existed as the head of the intelligence community, rather than the CIA which had previously served as the titular head of the IC. EO 13555 partially supplemented and superseded EO 12333. On July 30, 2008, President George W. Bush signed EO 13470, which further supplemented and superseded EO 12333 to strengthen the role of the Director of National Intelligence.

Since the Snowden revaluations there has been a great deal of discussion regarding the activities of the IC community, but relatively little attention has been paid to EO 12333. EO 12333 often serves an alternate basis of authority for surveillance activities, above and beyond Section 215 and 702. As Bruce Schneier has emphasized, "Be careful when someone from the intelligence community uses the caveat "not under this program," or "not under this authority"; almost certainly it means that whatever it is they're denying is done under some other program or authority. So when[NSA General Counsel Raj] De said that companies knew about NSA collection under Section 702, it doesn't mean they knew about the other collection programs." Senator Dianne Feinstein (D-CA), Chair of the Senate Intelligence Committee, has said in August 2013 that, "The committee does not receive the same number of official reports on other NSA surveillance activities directed abroad that are conducted pursuant to legal authorities outside of FISA (specifically Executive Order 12333), but I intend to add to the committee's focus on those activities." In July 2014, a former Obama State Department official, John Napier Tye, wrote an Op-Ed in the Washington Post calling for greater scrutiny of EO 12333. Tye noted that "based in part on classified facts that I am prohibited by law from publishing, I believe that Americans should be even more concerned about the collection and storage of their communications under Executive Order 12333 than under Section 215."

Tye in the WaPo:

... [EO 12333] authorizes collection of the content of communications, not just metadata, even for U.S. persons. Such persons cannot be individually targeted under 12333 without a court order. However, if the contents of a U.S. person’s communications are “incidentally” collected (an NSA term of art) in the course of a lawful overseas foreign intelligence investigation, then Section 2.3(c) of the executive order explicitly authorizes their retention. It does not require that the affected U.S. persons be suspected of wrongdoing and places no limits on the volume of communications by U.S. persons that may be collected and retained.

[ E.g., NSA could "incidentally" retain the email of a US individual which happens to be mirrored in Google or Yahoo data centers outside the US, as part of bulk collection for an ongoing (never ending) foreign intelligence or anti-terrorism investigation... ]

“Incidental” collection may sound insignificant, but it is a legal loophole that can be stretched very wide. Remember that the NSA is building a data center in Utah five times the size of the U.S. Capitol building, with its own power plant that will reportedly burn $40 million a year in electricity.

See also Mining your data at NSA (source of image at top).

UPDATE: EO12333 + Obama OKs unprecedented sharing of this info as he leaves office = recent leaks? Note the use of the term "incidentally" and the wide dissemination (thanks to Obama policy change as he left office).

WSJ: ... “I recently confirmed that on numerous occasions the intelligence community incidentally collected information about U.S. citizens involved in the Trump transition,” Mr. Nunes said, reading a brief statement to reporters on Capitol Hill on Wednesday afternoon. “Details about U.S. persons associated with the incoming administration—details with little or no apparent foreign intelligence value—were widely disseminated in intelligence community reporting.”

... Mr. Nunes added that it was “possible” the president himself had some of his communication intercepted, and has asked the Federal Bureau of Investigation, National Security Agency and other intelligence agencies for more information.

The change put in place as Obama left office is probably behind the large number of circulating reports that feature "incidentally" captured communications of the Trump team. The NYTimes article below is from February.

NYTimes: ... Until now, National Security Agency analysts have filtered the surveillance information for the rest of the government. They search and evaluate the information and pass only the portions of phone calls or email that they decide is pertinent on to colleagues at the Central Intelligence Agency, the Federal Bureau of Investigation and other agencies. And before doing so, the N.S.A. takes steps to mask the names and any irrelevant information about innocent Americans.

The new system would permit analysts at other intelligence agencies to obtain direct access to raw information from the N.S.A.’s surveillance to evaluate for themselves. If they pull out phone calls or email to use for their own agency’s work, they would apply the privacy protections masking innocent Americans’ information — a process known as “minimization” — at that stage, Mr. Litt said.

... FISA covers a narrow band of surveillance: the collection of domestic or international communications from a wire on American soil, leaving most of what the N.S.A. does uncovered. In the absence of statutory regulation, the agency’s other surveillance programs are governed by rules the White House sets under a Reagan-era directive called Executive Order 12333.

... [it is unclear what] rules say about searching the raw data using names or keywords intended to bring up Americans’ phone calls or email that the security agency gathered “incidentally” under the 12333 surveillance programs ...

It appears that the number of individuals allowed to search bulk, incidentally collected data has been enlarged significantly. Who watches these watchers? (There must now be many thousands...)

Sophos: ... Patrick Toomey, a lawyer for the American Civil Liberties Union (ACLU), put it in an interview with the New York Times, 17 intelligence agencies are now going to be “rooting… through Americans’ emails with family members, friends and colleagues, all without ever obtaining a warrant”.

The new rules mean that the FBI, the CIA, the DEA, and intelligence agencies of the US military’s branches and more, will be able to search through raw signals intelligence (SIGINT): intercepted signals that include all manner of people’s communications, be it via satellite transmissions, phone calls and emails that cross network switches abroad, as well as messages between people abroad that cross domestic network switches.

Everything is Heritable

The figure above comes from the paper below. A quick glance shows that for pairs of individuals: 1. Increasing genetic similarity implies increasing trait similarity (for traits including height, cognitive ability, years of education) 2. Home environments (raised Together vs Apart; Adoptees) have limited impact on the trait (at least in relatively egalitarian Sweden).

It's all here in one simple figure, but still beyond the grasp of most people struggling to understand how humans and human society work... See also The Mystery of Non-Shared Environment.

Genetics and educational attainment

David Cesarini & Peter M. Visscher
NPJ Science of Learning 2, Article number: 4 (2017)
doi:10.1038/s41539-017-0005-6

Abstract: We explore how advances in our understanding of the genetics of complex traits such as educational attainment could constructively be leveraged to advance research on education and learning. We discuss concepts and misconceptions about genetic findings with regard to causes, consequences, and policy. Our main thesis is that educational attainment as a measure that varies between individuals in a population can be subject to exactly the same experimental biological designs as other outcomes, for example, those studied in epidemiology and medical sciences, and the same caveats about interpretation and implication apply.

Ginormous Neural Nets and Networks of Networks

Now that we have neural nets that are good at certain narrow tasks, such as image or speech recognition, playing specific games, translating language, ... the next stage of development will involve 1. linking these specialized nets together in a more general architecture ("Mixtures of Experts"), and 2. generalizing what is learned in one class of problems to different situations ("transfer learning"). The first paper below is by Google Brain researchers and the second from Google DeepMind.

See also A Brief History of the Future, as told to the Masters of the Universe.

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean
(Submitted on 23 Jan 2017)

The capacity of a neural network to absorb information is limited by its number of parameters. Conditional computation, where parts of the network are active on a per-example basis, has been proposed in theory as a way of dramatically increasing model capacity without a proportional increase in computation. In practice, however, there are significant algorithmic and performance challenges. In this work, we address these challenges and finally realize the promise of conditional computation, achieving greater than 1000x improvements in model capacity with only minor losses in computational efficiency on modern GPU clusters. We introduce a Sparsely-Gated Mixture-of-Experts layer (MoE), consisting of up to thousands of feed-forward sub-networks. A trainable gating network determines a sparse combination of these experts to use for each example. We apply the MoE to the tasks of language modeling and machine translation, where model capacity is critical for absorbing the vast quantities of knowledge available in the training corpora. We present model architectures in which a MoE with up to 137 billion parameters is applied convolutionally between stacked LSTM layers. On large language modeling and machine translation benchmarks, these models achieve significantly better results than state-of-the-art at lower computational cost.

PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, Daan Wierstra (Submitted on 30 Jan 2017)

For artificial general intelligence (AGI) it would be efficient if multiple users trained the same giant neural network, permitting parameter reuse, without catastrophic forgetting. PathNet is a first step in this direction. It is a neural network algorithm that uses agents embedded in the neural network whose task is to discover which parts of the network to re-use for new tasks. Agents are pathways (views) through the network which determine the subset of parameters that are used and updated by the forwards and backwards passes of the backpropogation algorithm. During learning, a tournament selection genetic algorithm is used to select pathways through the neural network for replication and mutation. Pathway fitness is the performance of that pathway measured according to a cost function. We demonstrate successful transfer learning; fixing the parameters along a path learned on task A and re-evolving a new population of paths for task B, allows task B to be learned faster than it could be learned from scratch or after fine-tuning. Paths evolved on task B re-use parts of the optimal path evolved on task A. Positive transfer was demonstrated for binary MNIST, CIFAR, and SVHN supervised learning classification tasks, and a set of Atari and Labyrinth reinforcement learning tasks, suggesting PathNets have general applicability for neural network training. Finally, PathNet also significantly improves the robustness to hyperparameter choices of a parallel asynchronous reinforcement learning algorithm (A3C).

The figure below describes the speedup in learning new games based on previous learning from playing a game of different type.

Dalton Conley: The Bell Curve Revisited and The Genome Factor

Dalton Conley is the Henry Putnam University Professor of Sociology at Princeton University. He is unique in having earned a second PhD in behavior genetics after his first in Sociology.

In the talk and paper below he discusses molecular genetic tests of three hypotheses from Herrnstein and Murray's The Bell Curve: Intelligence and Class Structure in American Life. This much vilified book is indeed about intelligence and class structure, but almost entirely not about race. Racial differences in intelligence are only discussed in one chapter, and the authors do not make strong claims as to whether the causes for these differences are genetic or environmental. (They do leave open the possibility of a genetic cause for part of the gap, which has led to all kinds of trouble for the surviving author, Charles Murray.) The three questions addressed by Dalton do not involve race.

Harvard professor Harvey Mansfield organized a panel to commemorate the 20th anniversary of The Bell Curve back in 2014. You can find the video here.

1. How is it that the "core propositions" of The Bell Curve can be discussed in a paper published in Sociological Science and at an advanced seminar at Princeton, but Charles Murray is not allowed to speak at Middlebury College?

2. There must be many social scientists or academics in the humanities (or undergraduates at Middlebury) with strong opinions about The Bell Curve, despite never having read it, and despite having a completely erroneous understanding of what the book is about. If you are one of these people, shouldn't you feel embarrassed or ashamed?

The Bell Curve Revisited: Testing Controversial Hypotheses with Molecular Genetic Data

Dalton Conley, Benjamin Domingue

Sociological Science, July 5, 2016
DOI 10.15195/v3.a23

In 1994, the publication of Herrnstein’s and Murray’s The Bell Curve resulted in a social science maelstrom of responses. In the present study, we argue that Herrnstein’s and Murray’s assertions were made prematurely, on their own terms, given the lack of data available to test the role of genotype in the dynamics of achievement and attainment in U.S. society. Today, however, the scientific community has access to at least one dataset that is nationally representative and has genome-wide molecular markers. We deploy those data from the Health and Retirement Study in order to test the core series of propositions offered by Herrnstein and Murray in 1994. First, we ask whether the effect of genotype is increasing in predictive power across birth cohorts in the middle twentieth century. Second, we ask whether assortative mating on relevant genotypes is increasing across the same time period. Finally, we ask whether educational genotypes are increasingly predictive of fertility (number ever born [NEB]) in tandem with the rising (negative) association of educational outcomes and NEB. The answers to these questions are mostly no; while molecular genetic markers can predict educational attainment, we find little evidence for the proposition that we are becoming increasingly genetically stratified.

While I find the work described above to be commendable (i.e., it foreshadows how molecular genetic methods will eventually address even the most complex and controversial topics in social science), I don't feel that the conclusions reached are beyond question. For example, see this worthwhile comment at the journal web page:

This is a fascinating study! My comments below pertain to Proposition #1, that “The effect of genetic endowment is increasing over time with the rise of a meritocratic society”.

The data reported here do not seem to unequivocally contravene H&M’s hypothesis. The authors focus on the interaction terms, PGS x Birth Year (i.e. cohort), and show that interaction coefficient is slightly negative (b=-0.006, p=0.47), indicating a weakening of the association between genetic endowment and educational attainment, broadly conceived. The finding is that PGSs (polygenic scores) are (slightly) less predictive of educational attainment in later cohorts.

This isn’t that surprising, given educational inflation – over time, higher percentages of the population achieve any given level of educational attainment. In addition, as shown in Table 3 and mentioned in the Discussion section, this decline in importance of genetic endowment is restricted only to the ‘lower half of the educational distribution’. In contrast, genetic endowment (measured by PGSs) has become even more important across cohorts in predicting the ‘transition from a completed college degree to graduate education’ (534). Isn’t this what we’d expect to happen as the level of educational attainment at the lower half of the distribution becomes increasingly decoupled from cognitive ability?

H&M argued that cognitive ability is becoming more important in determining one’s life chances. The authors of this paper don’t actually test this hypothesis. They instead create polygenic scores *of educational attainment* (!) rather than cognitive ability – based on the GWAS of Rietveld et al. (2013) – and find that genetic predictors of *educational attainment* become (slightly) less predictive of educational attainment, on average, i.e. for high school and college. But again, they also report that the association of this genetic correlate (of educational attainment) and educational attainment has actually become stronger for transitions into graduate and professional schools from college.

If I’m not mistaken, the association between cognitive ability (as measured say by standardized tests, SAT, ACT, GRE, AFQT and NEA reports on reading and math ability) and educational attainment has weakened over time. It is possible that cognitive ability is becoming increasingly salient in determining SES as H&M maintain, and at the same time, educational attainment is becoming less salient, simply because the relationship between cognitive ability and educational attainment is becoming weaker. In other words, educational attainment, at least at the lower levels, is less salient in determining relative status. ...

Regarding fertility and dysgenic trends, see this more recent paper from the DeCode collaboration in Iceland, which reaches much stronger conclusions in agreement with H&M.

See also Conley's new book Genome Factor: What the Social Genomics Revolution Reveals about Ourselves, Our History, and the Future.

Conley, Steve Pinker, and I were on a 92nd Street Y panel together in 2016.

"We need to encourage real diversity of thought in the professoriate"

John Etchemendy is a former Provost of Stanford University.

The Threat From Within

... Over the years, I have watched a growing intolerance at universities in this country – not intolerance along racial or ethnic or gender lines – there, we have made laudable progress. Rather, a kind of intellectual intolerance, a political one-sidedness, that is the antithesis of what universities should stand for. It manifests itself in many ways: in the intellectual monocultures that have taken over certain disciplines; in the demands to disinvite speakers and outlaw groups whose views we find offensive; in constant calls for the university itself to take political stands. We decry certain news outlets as echo chambers, while we fail to notice the echo chamber we’ve built around ourselves.

This results in a kind of intellectual blindness that will, in the long run, be more damaging to universities than cuts in federal funding or ill-conceived constraints on immigration. It will be more damaging because we won’t even see it: We will write off those with opposing views as evil or ignorant or stupid, rather than as interlocutors worthy of consideration. We succumb to the all-purpose ad hominem because it is easier and more comforting than rational argument. But when we do, we abandon what is great about this institution we serve.

It will not be easy to resist this current. As an institution, we are continually pressed by faculty and students to take political stands, and any failure to do so is perceived as a lack of courage. But at universities today, the easiest thing to do is to succumb to that pressure. What requires real courage is to resist it. Yet when those making the demands can only imagine ignorance and stupidity on the other side, any resistance will be similarly impugned.

The university is not a megaphone to amplify this or that political view, and when it does it violates a core mission. Universities must remain open forums for contentious debate, and they cannot do so while officially espousing one side of that debate.

But we must do more. We need to encourage real diversity of thought in the professoriate, and that will be even harder to achieve. It is hard for anyone to acknowledge high-quality work when that work is at odds, perhaps opposed, to one’s own deeply held beliefs. But we all need worthy opponents to challenge us in our search for truth. It is absolutely essential to the quality of our enterprise.

I fear that the next few years will be difficult to navigate. We need to resist the external threats to our mission, but in this, we have many friends outside the university willing and able to help. But to stem or dial back our academic parochialism, we are pretty much on our own. The first step is to remind our students and colleagues that those who hold views contrary to one’s own are rarely evil or stupid, and may know or understand things that we do not. It is only when we start with this assumption that rational discourse can begin, and that the winds of freedom can blow.

See also

Why Universities Must Choose One Telos: Truth or Social Justice

by Jonathan Haidt | Oct 21, 2016

Aristotle often evaluated a thing with respect to its “telos” – its purpose, end, or goal. The telos of a knife is to cut. The telos of a physician is health or healing. What is the telos of university?

The most obvious answer is “truth” –- the word appears on so many university crests. But increasingly, many of America’s top universities are embracing social justice as their telos, or as a second and equal telos. But can any institution or profession have two teloses (or teloi)? What happens if they conflict?

As a social psychologist who studies morality, I have watched these two teloses come into conflict increasingly often during my 30 years in the academy. The conflicts seemed manageable in the 1990s. But the intensity of conflict has grown since then, at the same time as the political diversity of the professoriate was plummeting, and at the same time as American cross-partisan hostility was rising. ...

Examples of Perverse Incentives and Replication in Science

In an earlier post (Perverse Incentives and Replication in Science), I wrote:

Here's a depressing but all too common pattern in scientific research:

1. Study reports results which reinforce the dominant, politically correct, narrative.
2. Study is widely cited in other academic work, lionized in the popular press, and used to advance real world agendas.
3. Study fails to replicate, but no one (except a few careful and independent thinkers) notices.

This seems to have hit a nerve, as many people have come forward with their own examples of this pattern.

From a colleague at MSU:

Parts 1 and 2: Green revolution in Malawi from Farm Input Subsidy Program? Hurrah! Gushing coverage in the NYTimes, Jeffrey Sachs claiming credit, etc.

http://www.nytimes.com/2012/04/20/opinion/how-malawi-fed-its-own-people.html
http://www.nytimes.com/2007/12/02/world/africa/02malawi.html
http://www.nytimes.com/slideshow/2007/12/01/world/20071202MALAWI_index.html

Part 3: Failed replication? No actual green revolution? Will anyone notice?

Re-evaluating the Malawian Farm Input Subsidy Programme (Nature Plants)

Joseph P. Messina1*†, Brad G. Peter1† and Sieglinde S. Snapp2†

Abstract: The Malawian Farm Input Subsidy Programme (FISP) has received praise as a proactive policy that has transformed the nation’s food security, yet irreconcilable differences exist between maize production estimates distributed by the Food and Agriculture Organization of the United Nations (FAO), the Malawi Ministry of Agriculture and Food Security (MoAFS) and the National Statistical Office (NSO) of Malawi. These differences illuminate yield-reporting deficiencies and the value that alternative, politically unbiased yield estimates could play in understanding policy impacts. We use net photosynthesis (PsnNet) as an objective source of evidence to evaluate production history and production potential under a fertilizer input scenario. Even with the most generous harvest index (HI) and area manipulation to match a reported error, we are unable to replicate post-FISP production gains. In addition, we show that the spatial delivery of FISP may have contributed to popular perception of widespread maize improvement. These triangulated lines of evidence suggest that FISP may not have been the success it was thought to be. Lastly, we assert that fertilizer subsidies may not be sufficient or sustainable strategies for production gains in Malawi.

Introduction: Input subsidies targeting agricultural production are frequent and contentious development strategies. The national scale FISP implemented in Malawi has been heralded as an ‘African green revolution’ success story1. The programme was developed by the Malawian government in response to long-term recurring food shortages, following the notably poor maize harvest of 2005; the history of FISP is well described by Chirwa and Dorward2. Scholars and press sources alike commonly refer to government statistics regarding production and yields as having improved significantly. Reaching widespread audiences, Sachs broadcasted that “production doubled within one harvest season” following its deployment3. The influential policy paper by Denning et al. opened with the statement that the “Government of Malawi implemented one of the most ambitious and successful assaults on hunger in the history of the African continent”4. The Malawi success narrative has certainly influenced global development agencies, resulting in increased support for agricultural input subsidies; Tanzania, Zambia, Kenya and Rwanda have all followed suit and implemented some form of input subsidy programme. There has been mild economic criticism of the subsidy implementation process, including disruption of private fertilizer distribution networks within the policy’s first year5. Moreover, the sustainability of subsidies in Malawi has been debated6,7, yet crop productivity gains from subsidies have gone largely unquestioned. As Sanchez commented, “in spite of criticisms by donor agencies and academics, the seed and fertilizer subsidies provided food security to millions of Malawians”1. This optimistic assessment of potential for an “African green revolution” must be tempered by the fact that the Malawian production miracle appears, in part, to be a myth. ...

For more on the 1-2-3 pattern and replication, see this blog post by economist Douglas Campbell, and discussion here.

For more depressing narrative concerning the reliability of published results (this time in cancer research), see this front page NYTimes story from today.

The Eyes of Texas

Sorry for the blogging interruption. I'm at the annual AAU (Association of American Universities) meeting of Senior Research Officers in Austin, Texas.

UT Austin has a beautiful clock tower just up the street from our hotel. As pretty as it is I couldn't help but think about the 1966 tower sniper (45 casualties in 96 minutes) while walking around the main quad. It's a heartbreaking story.

The Eyes of Texas are upon you,
All the live long day.
The Eyes of Texas are upon you,
You can not get away.
Do not think you can escape them
At night or early in the morn
The Eyes of Texas are upon you
'Till Gabriel blows his horn.

Perverse Incentives and Replication in Science

Here's a depressing but all too common pattern in scientific research:

1. Study reports results which reinforce the dominant, politically correct, narrative.

2. Study is widely cited in other academic work, lionized in the popular press, and used to advance real world agendas.

3. Study fails to replicate, but no one (except a few careful and independent thinkers) notices.

For numerous examples, see, e.g., any of Malcolm Gladwell's books :-(

A recent example: the idea that collective intelligence of groups (i.e., ability to solve problems and accomplish assigned tasks) is not primarily dependent on the cognitive ability of individuals in the group.

It seems plausible to me that by adopting certain best practices for collaboration one can improve group performance, and that diversity of knowledge base and personal experience could also enhance performance on certain tasks. But recent results in this direction were probably oversold, and seem to have failed to replicate.

James Thompson has given a good summary of the situation.

Parts 1 and 2 of our story:

MIT Center for Collective Intelligence: ... group-IQ, or “collective intelligence” is not strongly correlated with the average or maximum individual intelligence of group members but is correlated with the average social sensitivity of group members, the equality in distribution of conversational turn-taking, and the proportion of females in the group.

Is it true? The original paper on this topic, from 2010, has been cited 700+ times. See here for some coverage on this blog when it originally appeared.

Below is the (only independent?) attempt at replication, with strongly negative results. The first author is a regular (and very insightful) commenter here -- I hope he'll add his perspective to the discussion. Have we reached part 3 of the story?

Smart groups of smart people: Evidence for IQ as the origin of collective intelligence in the performance of human groups

Timothy C. Bates a,b,⁎, Shivani Gupta a
a Department of Psychology, University of Edinburgh
b Centre for Cognitive Ageing and Cognitive Epidemiology, University of Edinburgh

What allows groups to behave intelligently? One suggestion is that groups exhibit a collective intelligence accounted for by number of women in the group, turn-taking and emotional empathizing, with group-IQ being only weakly-linked to individual IQ (Woolley, Chabris, Pentland, Hashmi, & Malone, 2010). Here we report tests of this model across three studies with 312 people. Contrary to prediction, individual IQ accounted for around 80% of group-IQ differences. Hypotheses that group-IQ increases with number of women in the group and with turn-taking were not supported. Reading the mind in the eyes (RME) performance was associated with individual IQ, and, in one study, with group-IQ factor scores. However, a well-fitting structural model combining data from studies 2 and 3 indicated that RME exerted no influence on the group-IQ latent factor (instead having a modest impact on a single group test). The experiments instead showed that higher individual IQ enhances group performance such that individual IQ determined 100% of latent group-IQ. Implications for future work on group-based achievement are examined.

From the paper:

Given the ubiquitous importance of group activities (Simon, 1997) these results have wide implications. Rather than hiring individuals with high cognitive skill who command higher salaries (Ritchie & Bates, 2013), organizations might select-for or teach social sensitivity thus raising collective intelligence, or even operate a female gender bias with the expectation of substantial performance gains. While the study has over 700 citations and was widely reported to the public (Woolley, Malone, & Chabris, 2015), to our knowledge only one replication has been reported (Engel, Woolley, Jing, Chabris, & Malone, 2014). This study used online (rather than in-person) tasks and did not include individual IQ. We therefore conducted three replication studies, reported below.

... Rather than a small link of individual IQ to group-IQ, we found that the overlap of these two traits was indistinguishable from 100%. Smart groups are (simply) groups of smart people. ... Across the three studies we saw no significant support for the hypothesized effects of women raising (or men lowering) group-IQ: All male, all female and mixed-sex groups performed equally well. Nor did we see any relationship of some members speaking more than others on either higher or lower group-IQ. These findings were weak in the initial reports, failing to survive incorporation of covariates. We attribute these to false positives. ... The present findings cast important doubt on any policy-style conclusions regarding gender composition changes cast as raising cognitive-efficiency. ...

In conclusion, across three studies groups exhibited a robust cognitive g-factor across diverse tasks. As in individuals, this g-factor accounted for approximately 50% of variance in cognition (Spearman, 1904). In structural tests, this group-IQ factor was indistinguishable from average individual IQ, and social sensitivity exerted no effects via latent group-IQ. Considering the present findings, work directed at developing group-IQ tests to predict team effectiveness would be redundant given the extremely high utility, reliability, validity for this task shown by individual IQ tests. Work seeking to raise group-IQ, like re- search to raise individual IQ might find this task achievable at a task- specific level (Ritchie et al., 2013; Ritchie, Bates, & Plomin, 2015), but less amenable to general change than some have anticipated. Our attempt to manipulate scores suggested that such interventions may even decrease group performance. Instead, work understanding the developmental conditions which maximize expression of individual IQ (Bates et al., 2013) as well as on personality and cultural traits supporting cooperation and cumulation in groups should remain a priority if we are to understand and develop cognitive ability. The present experiments thus provide new evidence for a central, positive role of individual IQ in enhanced group-IQ.

Meta-Observation: Given the 1-2-3 pattern described above, one should be highly skeptical of results in many areas of social science and even biomedical science (see link below). Serious researchers (i.e., those who actually aspire to participate in Science) in fields with low replication rates should (as a demonstration of collective intelligence!) do everything possible to improve the situation. Replication should be considered an important research activity, and should be taken seriously.

Most researchers I know in the relevant areas have not yet grasped that there is a serious problem. They might admit that "some studies fail to replicate" but don't realize the fraction might be in the 50 percent range!

More on the replication crisis in certain fields of science.