Do you use tabs or spaces for code indentation?
This is a bit of a “holy war” among software developers; one that’s been the subject of many debates and in-jokes. I use spaces, but I never thought it was particularly important. But today we’re releasing the raw data behind the Stack Overflow 2017 Developer Survey, and some analysis suggests this choice matters more than I expected.
Spaces make more money than tabs
There were 28,657 survey respondents who provided an answer to tabs versus spaces and who considered themselves a professional developer (as opposed to a student or former programmer). Within this group, 40.7% use tabs and 41.8% use spaces (with 17.5% using both). Of them, 12,426 also provided their salary.
Analyzing the data leads us to an interesting conclusion. Coders who use spaces for indentation make more money than ones who use tabs, even if they have the same amount of experience:
Indeed, the median developer who uses spaces had a salary of $59,140, while the median tabs developer had a salary of $43,750. (Note that all the results were converted into US dollars from each respondent’s currency). Developers who responded “Both” were generally indistinguishable from ones who answered “Tabs”: I’ll leave them out of many of the remaining analyses.
This is an amusing result, but of course it’s not conclusive by itself. When I first discovered this effect, I assumed that it was confounded by a factor such as country or programming language. For example, it’s conceivable that developers in low GDP-per-capita countries could be more likely to use tabs, and therefore such developers tend to have lower salaries on average.
We could examine this by considering whether the effect occurs within each country, for several of the countries that had the most survey respondents.
The effect is smaller in Europe and especially large in India, but it does appear within each country, suggesting this isn’t the sole confounding factor.
As another hypothesis, we know that different types of developers often use different indentation (e.g. with DevOps developers more likely to use spaces and mobile developers more likely to use tabs), often because they use different editors and languages. The Developer Survey asked both about what programming languages each respondent uses (Python, Javascript, etc) and what “type” of developer they are (web developer, embedded developer, etc).
Did we see the same tabs/spaces gap within each of these groups?
Yes, the effect existed within every subgroup of developers. (This gave a similar result even when filtering for developers only in a specific country, or for ones with a specific range of experience). Note that respondents could select multiple languages, so each of these groups are overlapping to some degree.
I did several other visual examinations of possible confounding factors (such as level of education or company size), and found basically the same results: spaces beat tabs within every group. Now that the raw data is available, I encourage other statisticians to check other confounders themselves.
Estimating the effect
If we control for all of the factors that we suspect could affect salary, how much effect does the choice of tabs/spaces have?
To answer this, I fit a linear regression, predicting salary based on the following factors.
- Tabs vs spaces
- Country
- Years of programming experience
- Developer type and language (for the 49 responses with at least 200 “yes” answers)
- Level of formal education (e.g. bachelor’s, master’s, doctorate)
- Whether they contribute to open source
- Whether they program as a hobby
- Company size
The model estimated that using spaces instead of tabs leads to a 8.6% higher salary (confidence interval (6%, 10.4%), p-value < 10^-10). (By predicting the logarithm of the salary, we were able to estimate the % change each factor contributed to a salary rather than the dollar amount). Put another way, using spaces instead of tabs was worth as much as an extra 2.4 years of experience.
Conclusion
So… this is certainly a surprising result, one that I didn’t expect to find when I started exploring the data. And it is impressively robust even when controlling for many confounding factors. As an exercise I tried controlling for many other confounding factors within the survey data beyond those mentioned here, but it was difficult to make the effect shrink and basically impossible to make it disappear.
Correlation is not causation, and we can never be sure that we’ve controlled for all the confounding factors present in a dataset. If you’re a data scientist, statistician, or analyst, I encourage you to download the raw survey data and examine it for yourself. You can find the code behind this blog post here if you’d like to reproduce the analysis. In any case we’d be interested in hearing hypotheses about this relationship.
Though for the sake of my own salary, I’m sticking with spaces for now.
-
Keith Hultman
-
Pasquale
-
Leonid Boytsov
-
-
Draco18s
-
Anthony Plant
-
-
Eric Hydrick
-
Vidhyashankar Madheswaraswamy
-
Anthony Plant
-
Nick Odaemus
-
-
Václav Haisman
-
Marius Gedminas
-
Chris Charabaruk
-
CeeBee
-
-
Václav Haisman
-
-
-
Chris
-
Izkata
-
CeeBee
-
-
Victor Lewis
-
Nick G
-
Martin Bean
-
-
Lundin
-
Jit
-
Lundin
-
Mike
-
Eric S. Bullington
-
Mike
-
-
klmr
-
-
Mike
-
-
Bryan Oakley
-
-
sqd
-
Izkata
-
POOL-POG
-
Pluckerpluck
-
CeeBee
-
Izkata
-
-
POOL-POG
-
-
Izkata
-
POOL-POG
-
CeeBee
-
-
-
Pluckerpluck
-
CeeBee
-
Bryan Oakley
-
CeeBee
-
-
-
-
Sik
-
Jason Sebring
-
CeeBee
-
-
zer0 0ne
-
GrandOpener
-
Thomas Peeters
-
Kratoklastes
-
klmr
-
Brian
-
-
-
commenter
-
Spacer
-
commenter
-
-
-
Marlon Stevenson
-
Daniel Samuels
-
Juank
-
Daniel Samuels
-
Sven Slootweg
-
Daniel Samuels
-
Bryan Oakley
-
-
chugly
-
CeeBee
-
-
Bryan Oakley
-
-
Bryan Oakley
-
Marlon Stevenson
-
-
Spacer
-
-
wurtel
-
Félix Gagnon-Grenier
-
CeeBee
-
Félix Gagnon-Grenier
-
CeeBee
-
Félix Gagnon-Grenier
-
CeeBee
-
-
-
wurtel
-
-
-
Max Savin
-
Boris Lazarov
-
Rolnard
-
Michael Beskin
-
David Lawrence
-
Michael Beskin
-
CeeBee
-
-
-
-
Jason Sebring
-
Sven Slootweg
-
Jason Sebring
-
Sven Slootweg
-
-
-
Bryan
-
commenter
-
Jason Sebring
-
-
-
Rolnard
-
Félix Gagnon-Grenier
-
CeeBee
-
-
Joey Robert
-
Chris Charabaruk
-
Joey Robert
-
Anthony Clink
-
-
-
acommenter
-
chugly
-
-
Petar Donchev
-
Anthony Clink
-
The Guy with The Hat
-
-
carrie
-
Benjamin H
-
Sven Slootweg
-
Benjamin H
-
Stilgar
-
Benjamin H
-
klmr
-
Thomas
-
Stilgar
-
-
Thomas
-
Sven Slootweg
-
chugly
-
-
-
Giorgio Polvara
-
Anthony Clink
-
Giorgio Polvara
-
CeeBee
-
-
-
-
Tyler Hibbard
-
Mike Barlow
-
Bryan Oakley
-
-
Bryan Oakley
-
-
Victoria
-
xjlin0
-
Ryan Thompson
-
-
Edwin Ramirez
-
Thomas
-
Alfonso Pérez
-
Inquisitor
-
Tyler Hibbard
-
klmr
-
Tyler Hibbard
-
Félix Gagnon-Grenier
-
-
-
Bryan Oakley
-
Félix Gagnon-Grenier
-
-
-
Bryan Oakley
-
-
greyenlightenment
-
caitlynmayers
-
Jan
-
Tyler Hibbard
-
Bryan Oakley
-
CeeBee
-
-
Tim!
-
-
-
-
DeeSnood
-
Tyler Hibbard
-
jyounker
-
CeeBee
-
-
Mike Barlow
-
Tyler Hibbard
-
Mike Barlow
-
-
-
Bryan Oakley
-
-
Matei Gabriel Copot
-
Ole Laursen
-
commenter
-
CeeBee
-
Hernan Silva
-
Bryan Oakley
-
Kevin Cox
-
Tim!
-
-
Ole Laursen
-
CeeBee
-
Ole Laursen
-
CeeBee
-
CeeBee
-
-
-
-
Papito
-
Edwin
-
Kevin Cox
-
tuba.terry
-
TennSeven
-
-
-
Pax
-
Tyler Hibbard
-
jdkolassa
-
Tim McClarren
-
Bryan Oakley
-
David Robinson
-
CeeBee
-
David Robinson
-
CeeBee
-
-
-
-
tuba.terry
-
-
Roger Creasy
-
minus Seven
-
Kyle Strand
-
-
-
Tony Adams
-
Lincoln
-
Frank
-
Simone
-
Václav Haisman
-
-
-
Félix Gagnon-Grenier
-
CeeBee
-
Félix Gagnon-Grenier
-
CeeBee
-
Paul Ishak
-
CeeBee
-
Paul Ishak
-
CeeBee
-
Paul Ishak
-
tuba.terry
-
Paul Ishak
-
commenter
-
Paul Ishak
-
commenter
-
Paul Ishak
-
Joshua Lamusga
-
commenter
-
Joshua Lamusga
-
commenter
-
Paul Ishak
-
Mario Rivera
-
Félix Gagnon-Grenier
-
CeeBee
-
-
-
Paul Ishak
-
CeeBee
-
-
-
Bones Justice
-
-
Samuel
-
Bryan Oakley
-
Chris G.
-
-
-
Frank
-
Bryan Oakley
-
Not Projecting
-
RadthorDax
-
-
-
Alex Artushin
-
commenter
-
Kostya Pogromskiy
-
commenter
-
Kostya Pogromskiy
-
commenter
-
-
-
-
Noel Hibbard
-
tuba.terry
-
-
Mark Lapasa
-
Marios Santamaria
-
-
TheBox193
-
RagnarDanneskjöld
-
Mike
-
-
camainc
-
Lynx
-
vim
-
jr1
-
-
sosiosh
-
TennSeven
-
-
Shirish
-
figital
-
Mike
-
Michael Clark
-
-
Daniel Hug
-
-
Dave Dyer
-
Kyle Strand
-
-
minus Seven
-
Scott Lavigne
-
s73v3r
-
Mike
-
Peter Graham
-
-
RadthorDax
-
Martijn
-
Kevin Smith
-
jr1
-
sheanmassey
-
Kevin Smith
-
-
CeeBee
-
-
Vrae
-
Jimmy
-
Iron Sean
-
Peter Graham
-
-
-
jr1
-
mmmpop
-
Dave
-
Kyle Strand
-
-
TechUser2011
-
CeeBee
-
tuba.terry
-
rumtscho
-
TennSeven
-
TechUser2011
-
TennSeven
-
-
-
-
Sagar Tanur
-
Iron Sean
-
Kevin Smith
-
-
Jack Johansson
-
Marius Wegner
-
Kyle Strand
-
-
Dennis Hort
-
sheanmassey
-
Jacob Stamm
-
John Reno
-
Nitin
-
-
sheanmassey
-
-
-
Adam Patterson
-
sheanmassey
-
Corey Wischmeyer
-
Red Cricket
-
jbw
-
sheanmassey
-
graywh
-
-
Rörd Hinrichsen
-
-
Sheikh Heera
-
Klaus Stock
-
Kyle Strand
-
-
Kyle Strand
-
-
Billy Hinners
-
Potato
-
Adam Lear
-
Jerry Coffin
-
Félix Gagnon-Grenier
-
-
Nutarama
-
-
Layton Miller
-
-
Stefan Lasiewski
-
Klaus Stock
-
Craig Allen
-
-
Anonymous Coward
-
Tristan Hyams
-
Stefan Lasiewski
-
-
stamsarger
-
Kyle Strand
-
GonzoI
-
-
Félix Gagnon-Grenier
-
Marco Ko
-
-
Marijana Larma
-
Chris G.
-
Félix Gagnon-Grenier
-
Adam Jorgensen
-
Stephen Ostermiller
-
Adam Jorgensen
-
-
-
-
NoSir
-
Chris G.
-
-
-
Red Cricket
-
Paulo Reichert
-
-
Erno
-
Chip Overclock®
-
Luna
-
GonzoI
-
Guilherme Taffarel Bergamin
-
GonzoI
-
-
-
-
rmonster
-
Guilherme Taffarel Bergamin
-
Zapados
-
-
Robert Boehne
-
GonzoI
-
-
tuba.terry
-
Guilherme Taffarel Bergamin
-
GonzoI
-
Guilherme Taffarel Bergamin
-
GonzoI
-
Guilherme Taffarel Bergamin
-
-
-
-
-
Opslag Medium
-
Guilherme Taffarel Bergamin
-
Chad Hansen
-
John
-
Guilherme Taffarel Bergamin
-
Nutarama
-
Guilherme Taffarel Bergamin
-
anonymous
-
-
-
-
-
Jay
-
GonzoI
-
Guilherme Taffarel Bergamin
-
-
Marco Ko
-
Marco Ko
-
Andrew Maxwell
-
Mike Lemmon
-
Cory Long
-
Tim!
-
Throstur T
-
-
Bill the Lizard
-
Félix Gagnon-Grenier
-
-
Josue Ibarra
-
Mike Darkins
-
Félix Gagnon-Grenier
-
-
Matteus Deloge
-
David Robinson
-
Nick
-
Matteus Deloge
-
anonymous
-
-
-
-
mingos
-
Yevgen Lasman
-
JadePenguin
-
Félix Gagnon-Grenier
-
el.dide
-
Félix Gagnon-Grenier
-
-
-
ryepdx
-
GonzoI
-
Félix Gagnon-Grenier
-
GonzoI
-
ozzy
-
-
ryepdx
-
GonzoI
-
-
Guilherme Taffarel Bergamin
-
GonzoI
-
Guilherme Taffarel Bergamin
-
Ext3h
-
Ext3h
-
GonzoI
-
Ext3h
-
-
Nick Brown
-
GonzoI
-
-
-
-
BlueNinjaSmurf
-
-
Colz
-
David Robinson
-
-
Arnold Spence
-
Nikhil Sahu
-
-
Ralf Kleberhoff
-
Kevin Norris
-
David Robinson
-
Kevin Norris
-
-
-
Baris Ozkuslar
-
Ritesh Rituraj Nayak
-
Stella Orion
-
Ritesh Rituraj Nayak
-
Stella Orion
-
-
-
-
GonzoI
-
Maxxon
-
Will
-
cb75075
-
Axel Heider
-
Brett Davis
-
Piotr Aueternum
-
-
handleym
-
Lex Irons
-
Operating Thetan
-
Morgan Grobin