Did you factor in the interviewees’ experience with interviewing.io, whether for the women it was a ‘last resort’ in their job search versus a ‘first resort’ for the men? That is, for the female interviewees, was this interview among the last in a long line of interviews, whereas for the male interviewees, was it among the first, if not the first, interview? I wondered if the tendency to use such a platform has this phenomenon as a difference between men and women in the job search process.

I really don’t care for this question man vs woman. Because if I cared it would probably meant that I am biasing for a side, which I don’t.
I know some women that kick some butts on development. They are awesome and they are widely recognize by their abilities.
But for every woman like this I know at least 5 men with same ability or better.

This is just my experience over 10 years.

Love all the people flailing their arms desperately looking for other ways discrimination was actually present. It’s got to be that the female-male modulator was worse quality! Or that speech patterns between men and women are different and this is where the discrimination happens!

Look ladies, if you have to keep coming up with more and more difficult to disprove means for discrimination each time evidence comes out supporting the opposite, you’re probably on the wrong side of the argument.

I’m wondering if some of the initial attrition rates between the sexes may be caused not by gender but by societal norms of gender in a particular region?

So I’m wondering if you would add a regional question into your analysis if it would show anything.

I’m also wondering if previous life experiences may also have a factor. Such as, what other activities do the interviewees participate in? Sports, Art, Music.

I’m not sure if this is bias on my part but where I am from it is more common for males to participate in athletics. I’m wondering if that may also be a factor as well, I say this because in athletics many are faced with failing but are commonly taught not to give up.

But this is also something that is taught in many extra curricular activities during someone’s education.

So when a person who uses vocal fry (like Aline), which is a linguistic trait highly associated with women, or other phrases, pauses, rhythm, and word choices that express femininity, has her voice modulated to sound like a man, she does slightly more poorly than other women? A person who sounds like a man but speaks like a woman does more poorly than a person who sounds like a woman and speaks like a woman.

But when a person has a voice that sounds female but his interests, phrasing, tone, rhythm, etc dog whistle to being male, he is rated slightly higher than a male voice that sounds like a man?

It seems to me that your test is showing a preference for male gender-based values, independent of pitch and biological sex. John Smith’s lamentable example above from the architecture and construction world is a great example of this bias. He has no value for the skills the women brought to the table in communication, making sure everyone’s needs were met, collaboration, sensitivity to the building’s use in it’s design and implementation, and so on. He judged the women’s social collaboration as making men do the hard decision making, with consensus being something he opposed as effeminate and therefore unprofessional. Women having emotions were deemed intentionally manipulative, conniving even. “Macho men,” in other words those with traditionally male behavior, however, were unable to “help themselves” in yielding to the women’s femininity. So he sympathizes with men responding in a traditionally male way while demonizing women acting in a traditionally female way and believes the workforce suffered from a positive integration of the feminine and masculine traits.

Would John Smith consider a poll of clients to see their satisfaction levels on projects with women or without women to see if end level satisfaction was higher, even if there were more corrections needed, when the communication and collaboration women brought to the table was a factor? He’d undoubtedly dismiss it as men unable to help themselves in condescendingly overrating the conniving women in order to avoid an emotional meltdown from expressing displeasure, as there couldn’t be an inherent value in something not in his own checklist of personality, traits, values, or skills.

I imagine if John Smith were one of your interviewers and he heard a woman who dog whistled to him through aggressive, egoistic, traditionally male phraseology, which is a male biological trait independent of culturalization, then he would take note and increase his measure of esteem for that woman who talks like a man (independent of pitch) above his general poor esteem for feminine traits in the workforce. He’s looking for someone to make the hard decisions by aggressive fiat, not someone who will go around consulting stakeholders to make sure consensus is reached. And, a person he perceived as male would likewise drop in his estimation when the man had the right pitch but talked like a woman, or like Aline with the vocal fry who “totally” did not sound “like a dude.”

Add in the wonderful point you uncovered about sensitivity leading to higher attrition among women, along with many of the great points others have made above (including, especially, the discouragement women get from being told they’re likely to face bias or fail–which there are studies to support both regarding gender and race), and the dynamics become multivariable and complex again. I agree that there’s a lot more that needs to be done.

I think it has more to do with culture. People in the nerdy world tend to fail at social cohesion with people who have interests outside of the nerdy world. This is true of any culture, but I think it’s worse in any profession historically male dominated. People who don’t feel they stand together with their peers tend to feel rejected. I think in combination with women being more sensitive to rejection and women tending to be the social butterfly it’s natural to want to find a career where it feels right. Which usually isn’t one where you need to be good at fending off doubts. My self identity is rebellious to expectations outside of my own and proud of my geeky-ness. I don’t struggle with that. I totally see it with other women. That feeling of rejection is less a bias and more a cultural gap that further feeds into a feeling of rejection.

A thought on your initial results of “no apparent change”. Maybe your assumption that voice modulation is a gender mask is incorrect? There might be subtle cues in language choice or communication style that are not affected by the voice modulation.

You might be able to determine effectiveness by asking the interviewer to guess interviewee gender at the end of the interview.

Thank you for posting this. It was an interesting read, and I look forward to seeing any future results as you continue to look at this.

I wanted to suggest that if we divide people into the (false but useful for these purposes) binary of women and men and look at average starting points on your theoretical attrition curve, we shouldn’t expect them to be in the same place. That is, by the time  women are at the stage of being interviewed for jobs, they have, on average, encountered quite a number more potential attrition events. I am a PhD candidate in a STEM field and I have always had a great love of mathematics. I started getting discouraging feedback even as a child, but there was plenty more to come in university. There are more formal results that suggest this is not an atypical experience for women, and that women encounter this type of negative feedback more often than men do in STEM. So one possibility is that women and men have the same attrition curve, but women are generally much further along it by the time they are at the interview stage. But even if it is not the case that the curve is the same, I would still expect women to be further along their curve. (Assuming, of course, that it even works to have such a curve. I recognise that in that part of the article you were more just musing on the topic and not presenting it as solid data, and I am similarly participating in this type of thinking out loud. It’s quite possible that a single curve like this lacks the right parameters to really model it.) I mean, hell, even just reading these comments there are far more people insisting that women just aren’t as good at tech than there are suggesting that men aren’t as good at tech. The worst position men seem to be put in is of being merely the same as women. What is the cumulative effect of seeing this in the comments of every piece on this topic? Does that push any women further along their attrition curve?

Another alternative is this:
If the disparity disappears once you weed out people who quit after one or two interviews, then perhaps it isn’t really about which gender dusts themselves off better but some other parameter that happens to have been correlated with gender for non-direct reasons. It’s possible you happen to have more women than men who are near that “final straw” point when entering the interviews, and that people already considering giving up don’t perform as well in interviews. Especially since for those remaining people the performance seems even across gender.

Something that isn’t clear to me:
When you say most people opted in for the the study, does it apply to both interviewers and interviewees? That is, did all of the interviewers opt in or did any choose not to? If there are any interviewers who didn’t participate, I would be concerned about the bias that would introduce. It could, for instance, select for interviewers who don’t see the merit in the experiment because they are fine with the gender gap or accept stereotype-based explanations for it, and maybe these people are more likely to have a negative impact in women’s rankings and performance. Or could introduce bias of some other origin.

One last bias thought:
Interviewers who know they are being evaluated might behave differently. I don’t mean they do it deliberately or with some intention to mislead, but that they suddenly have a heightened awareness of the issue and that might slightly shift how they behave because it is present in their thoughts. How did the subset of unmodulated interviewees compare to the modulated groups and to the pre-study numbers you produced? Any difference there?

To anyone in the comments who hasn’t read it, I highly recommend Cordelia Fine’s book “Delusions of Gender”. It presents, critiques, and synthesizes a fair bit of research around aptitude and gender taken from neuroscience and psychology. Perhaps counterintuitively, I especially wish to recommend the book to those who already support women in STEM, because it will provide you with published references that back up your explanations when talking to those who don’t want us around (who are much less likely to be willing to simply read the book). And there is enough material that even someone well read on the topic has a high likelihood of encountering some new studies or new takes on them. It’s quite accessibly written and also available as an audiobook and I swear I have no connection whatsoever to the author. 🙂

Upvote for “Tharrrrr be the gender gap”

First off, to make sure that there are no misconceptions about the points I will try to make below: When I talk about “men”, “women”, “boys”, “girls”, I talk about statistical means. I assume that the distributions I refer to are normally distributed, unless specified otherwise. On the other hand, that means that the individual person can be anywhere on that distribution.

“Specifically, men were getting advanced to the next round 1.4 times more often than women. Interviewee technical score wasn’t faring that well either — men on the platform had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women.

Despite these numbers, it was really difficult for me to believe that women were just somehow worse at computers, so when some of our customers asked us to build voice masking to see if that would make a difference in the conversion rates of female candidates, we didn’t need much convincing.”

People are different. People have different abilities and interests. Obviously, men and women are also different. Seeing that men and women are already different superficial (i.e. they look different), wouldn’t it be fair to assume they “function” differently, i.e. have different interests and abilities?

Then why is it hard to accept that men might simply be more apt in computer related topics? Why should that conjecture be specifically excluded? It explains everything, AND it matches observations. Men might simply be more interested in technical stuff than women. Why is that bad? I still fail to see the point of the social engineering idea that instead of having people do stuff they are apt in AND enjoy, to push them into something they might not. Studies have shown that women in rich countries tend to be LESS likely to go for technical jobs, BECAUSE the economic and social security enables them to do what THEY want. So also data proves that women apparently simply do not want to deal with technical issues, if they can freely choose not to. That is sufficient to explain the employment gap in STEM (in western societies). I sincerely ask why we should ignore that huge elephant in the room and try to make up experiments to find evidence for a conjecture which is created specifically from the mindset “but, that cannot be!”. Maybe it can be and is so, and that’s fine.

Why do we “need” more women in STEM? In my job, I want to have colleagues who know what they’re doing. What gender they have is not important for that. And if it so happens that women are not interested and maybe not equally able do work in tech, what’s the big deal? I think the real issue in western societies is that jobs which are actually supporting the welfare and stability of the society are done to a large extent by women, and are massively undervalued (by pay, which also explains the mythical gender pay gap, which doesn’t account for job fields etc.). That’s the issue that needs fixing. Fighting to gain a foothold in STEM, because apparently wages are higher there misses the point. What women and feminists should fight for, is that jobs women are actually interested in and doing to be valued much higher than they are right now. That would be a fight which goes hand in hand with the interest of women (i.e. the bulk of the distribution), and not the small fraction which is interested and able to work in STEM.

Fight for better pay and conditions in hospitals, kindergartens, schools, etc! The rich investor can only do his job because there is a huge workforce of underpaid people who keep the real life of his back, so that he can play casino at the stock markets. Show them who’s really running daily life!

I remember hearing about interviewing.io on NPR. I think the idea is great but the execution is very poor. The pitch of a “male” or a “female” voice isn’t the only distinguishing characteristic. Men and women simply speak differently: inflection, intensity, duration, diction, etc. etc. etc. make it pretty easy to distinguish a male speaker from a female speaker, even when the voice is “masked.” In all three interviewing.io voice-modulated examples I’ve heard, it’s been pretty easy to tell if the speaker is male or female (yes, without knowing the speaker’s gender beforehand). I would be surprised if most, if not all, interviewers didn’t figure out the gender of the interviewee pretty early on in the conversation. I’m wondering if interviewing.io is more a good PR move for employers rather than an effective tool for increasing diversity in the workplace. Has interviewing.io considered this? Have you done any research on it? (Play a few voice-modulated examples for a focus group and ask them to guess the speaker’s gender?)

There is a nice documentary about how this gender gap is mostly because of biology
One of the studies found that how wealthier the country is, how bigger the gender gap will be because people will study the things that they find intresting, while in poor countries (uganda, ghana) the gender gap decreases because the technical jobs will have a bigger chance at a better life

link to the documentary

Purely conjecture, but could this issue be something that starts even at the beginning of Computer Science education? Do women tend to give up on CS more than men when struggling, resulting in a gender gap that grows wider and wider as you progress through the education levels?

This could possibly explain why the industry has a large gender gap, because so many are weeded out before they even sit down for an interview.

Obviously, we need to lower standards and impose hiring quotas for women in this sector. Otherwise, any other course of action would be misogynistic and biased.

The last thing we would want to do would to be hold women accountable for their actions and ascribe to them some degree of agency in their decisions to give up early in this context.

That would be sexist. They aren’t decisive, free-willed humans after all. They are women fighting the all-invasive Patriarchy which robs them of agency and accountability.

I believe gender impacts how we interview by more than just the sound of our voices. Women behind voice modulation that causes them to sound like men may still lack the swagger/confidence in their interview that men often exhibit, leading one to believe they are less qualified than their male counterparts. Because we are learning to account for that as interviewers, I wonder if the voice modulation does actively hurt women’s perceived performance.

I’m sure people will love talking to the witness protection program voice during interviews. Maybe for live coding exercises you can put the candidates in a dimly lit room and put a blur effect over their faces.

Come on man, this is ridiculous and you know it.

“Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance”

No, in fact, that’s exactly what I expected.

While it’s convenient to think of the world in black-and-white, that’s not often a true model of reality. To believe that voice tone matters is to believe that men are systematically hearing a female voice and ranking it less competent, i.e., that there are simply some significant number of people in this world who are unapologetic sexists.

The main problem with this is that most tech interviewers are young enough that they went through college with a lot of women. These days, women are significantly more likely to attend college than men, so for anyone under about 40, their entire career experience is evidence against the belief that women are fundamentally incompetent. Why would we believe they would rank a female voice as less competent?

That didn’t used to be true. The “blatant sexist” can be an accurate model when interviewers are much older. For example, orchestra auditions are infamously sexist in this way (hence why blind auditions have seen some success). Up until fairly recently, conductors tended to be older men, who went to college back when not many women did.

Do you keep data on the sex of the interviewers themselves? I’m curious if their sex could have played any role in the candidates decision to drop out, and if those female candidates interviewed by female interviewers were less likely to drop out. I imagine that if the attrition is caused by females feeling intimidated by the sense of not belonging or worthiness as is theorized, that having a role model of their own gender might have helped to combat that.

“and why that difference matters for the thorny issue of the gender gap in tech.”

There is no gender gap, there is only qualified and unqualified. Artificially hiring women because, you know, women, under the guise of fighting this non-existent boogyman will only make “tech” worse.

Way to brainlessly perpetuate the myth though.