The thought first occurred to me, back in the fall of 2008, as I was reading Dan Ariely’s very readable and fascinating book, “Predictably Irrational”. Chapter after chapter is peppered with conclusions drawn from experiments conducted on students studying at some of the best institutions in the US. I wrote an email to a couple of the authors of what I thought were well-respected blogs about the brain and behavior. I wrote:
“The more I read about our cognitive biases or irrationality, the more I’m struck by how many decisions have been reached using what seem fairly limited samples, many of them just college students. I don’t doubt that we’ve cognitive biases but I wonder are there any studies that go across cultures, socio-economic strata and age in determining the cognitive biases ? I googled and couldn’t find anything relevant. Is it that we all have the same cognitive biases but different ones are brought to the fore by culture ? ”
I did not get any responses to my question (they probably were optimistic that I’d learn to google better). But the feeling never went away as I encountered many new fascinating conclusions drawn from experiments conducted on college students in this country. As I was writing my entry about the death of languages, I noticed an article in that excellent blog, Neuroanthropology. While the title caught my eye, I didn’t really go back to checking the article until a few days later. The entry is titled: “We agree it’s WEIRD, but is it WEIRD enough?“. WEIRD turns out to be an acronym for Western, Educated, Industrialized, Rich, and Democratic. The article was based on a paper (then) recently published paper titled The weirdest people in the world? by Joseph Henrich, Steven J. Heine and Ara Norenzayan.
There are two main points to this paper. The first is that a lot of conclusions in behavior science are based on experiments on WEIRD people, essentially undergrad students at Western, mostly American, universities. The second thrust of the paper is to show that WEIRD people are not representative of most of humanity when it comes to behaviors.
From the abstract of the paper:
“The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans. Many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior – hence, there are no obvious a priori grounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. Overall, these empirical patterns suggests that we need to be less cavalier in addressing questions of human nature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity.”
Two pieces of data from the article reflect the level of skewness in the papers related to behavior science:
- “A recent analysis of the top journals in six sub-disciplines of psychology from 2003 to 2007 revealed that 68% of subjects came from the United States, and a full 96% of subjects were from Western industrialized countries, specifically those in North America and Europe, as well as Australia and Israel (Arnett 2008). The make-up of these samples appears to largely reflect the country of residence of the authors, as 73% of first authors were at American universities, and 99% were at universities in Western countries. This means that 96% of psychological samples come from countries with only 12% of the world’s population.”
- “In the Journal of Personality and Social Psychology, the premier journal in social psychology – the subdiscipline of psychology that should (arguably) be the most attentive to questions about the subjects’ backgrounds – 67% of the American samples (and 80% of the samples from other countries) were composed solely of undergraduates in psychology courses (Arnett 2008).”
These papers and their conclusions are not just academic papers published in academic journals for the consumption of academics. As the authors write: “In top journals such as Nature and Science, researchers frequently extend their findings from undergraduates to the species – often declaring this generalization in their titles. These contributions typically lack even a cautionary footnote about these inferential extensions.”
The authors compare WEIRD people at four levels: western, industrialized countries vs what they term “small scale socieities”, Western industrialized countries vs non-Western, industrialized countries, American vs other western countries and finally university-educated Americans vs non-university-educated Americans. The authors base their comparisons on different aspects of behavior ranging from visual perception and spatial cognition to ideas of independence and inter-dependence and moral reasoning.
The main paper is a well written (I confess here that I only read sections of it, given my limited time and domain-specific competence) 22 pages or so. The reminder of the paper is a collection of responses from various peers to their paper and the authors’ response to the responses. Many of the responses apply the criticism to other areas such as neuroscience, linguistics, philosophy and the anthropocentric and ethnocentric attitudes related to comparing humans with other animals. Many responses question whether WEIRD is weird enough. For example, the author of the Neuroanthropology post writes:
“I worry that W.E.I.R.D. classification flatters the WEIRD, focusing on traits that Westerners typically highlight to describe themselves in ways that are, however inadvertently, pretty self-congratulatory. If we were to call the same group, Materialist, Young, self-Obsessed, Pleasure-seeking, Isolated, Consumerist, and Sedentary (MYOPICS)… you get the idea.”
A small section of the peer commentary argued against the conclusions of the article with one author even stating “WEIRD societies may be more compatible with human nature”.
In the main paper, the authors don’t attempt to explain the reasons behind the extreme differences in behavior of the WEIRD folks. But in their response to the various pieces of peer review, they touch upon this subject. They suggest two possible reasons. One is the primacy of the English language. They write: “ English-bias may be impacting theorizing in the cognitive sciences, while Machery and Stich show that it has impacted philosophical inquiry”.
The second cause they speculate has to do with the relative strangeness of American middle and upper class child-rearing techniques. They write: “Lancy lays the groundwork by highlighting the relative strangeness, in a broad global and historical context, of modern middle- and upper-class American beliefs, values, cultural models, and practices vis-a-vis childrearing. Fernald and Karasik et al. review evidence that is beginning to document how these practices impact cognitive, linguistic, and motor development, including long-term cognitive outcomes.”
We’ve been here before. Many times have we encountered the notion of researcher bias and skew caused by the nature of the samples studied.
I first encountered the idea of researcher bias many years ago when I was reading Dorothy Rowe’s insightful book, “Friends and Enemies: Our Need To Love and Hate”. She writes: “An American researcher observing a number of white, middle-class American babies, or an English researcher observing a number of white, middle-class English babies can easily fail to draw the simple conclusion that this is what American or English middle-class babies do and instead generalize their observations to say that this is what all babies do.”
A more tragic story is narrated by Robert Sapolsky in his essay “Poverty’s Remains” from his book, “The Trouble With Testosterone”. Much of our understanding of human anatomy was initially based on cadavers of poor people whose internal organs were differently shaped and sized because of the way they suffered and died. Sapolsky quotes several examples of this from the thymus gland which is very small in people who live in chronically stressful conditions to the adrenal gland which is much larger in people living under stressful conditions. So, at the turn of the 19th century, the doctors had a misconceived notion of the “normal” size of organs.
Before SIDS (Sudden Infant Death Syndrome) was called SIDS, an Austrian pathologist named Richard Paltauf concluded – after several autopsies of healthy infants who had died of unexplainable causes in their crib – that the cause of death was an enlarged thymus that pressed down on the trachea, strangling a sleeping infant. SIDS, which had been nameless thus far, was called status thymicolymphaticus and as far as into the 1950s, the preferred preventive treatment for SIDS was to irradiate the throats of infants. While not helping SIDS, the treatment resulted in causing thyroid cancer in tens of thousands of people. Sapolsky writes: “It is a chilling experience to wander the dusty lower floor of a medical library, reading forgotten seventy-year-old pediatric texts with their dry discussions of status thymicolymphaticus. The technical details of the disorder, the plausible etiology, the photographs of the “enlarged” thymuses, the confident recommendation for treatment – all wrong, page after page.”
More recently, at the start of the year, NYT published an article titled “The Americanization of Mental Illness”:
“AMERICANS, particularly if they are of a certain leftward-leaning, college-educated type, worry about our country’s blunders into other cultures. In some circles, it is easy to make friends with a rousing rant about the McDonald’s near Tiananmen Square, the Nike factory in Malaysia or the latest blowback from our political or military interventions abroad. For all our self-recrimination, however, we may have yet to face one of the most remarkable effects of American-led globalization. We have for many years been busily engaged in a grand project of Americanizing the world’s understanding of mental health and illness. We may indeed be far along in homogenizing the way the world goes mad.
This unnerving possibility springs from recent research by a loose group of anthropologists and cross-cultural psychiatrists. Swimming against the biomedical currents of the time, they have argued that mental illnesses are not discrete entities like the polio virus with their own natural histories. These researchers have amassed an impressive body of evidence suggesting that mental illnesses have never been the same the world over (either in prevalence or in form) but are inevitably sparked and shaped by the ethos of particular times and places.”
I cannot recommend the WEIRD paper highly enough to anybody engaged in an enquiry of human behavior, as a vocation or an avocation. I’ve saved a copy of the paper for a more thorough reading. I also recommend reading the post on Neuroanthropology for some additional insightful commentary.
Sapolsky concludes his essay “Poverty’s Remains” with a powerful and eloquent statement: “Be really certain before you ever pronounce something to be the norm, because at that instant, you have now made it supremely difficult to ever look at an exception to that supposed norm and see it objectively.”
