How (not) to measure democracy
Abstract
Measures of democracy are regularly employed in the statistical analysis of economic, political, and social policy. This paper reviews the measures" setup, strength, and weaknesses across the three most prominent democracy datasets: PolityIV, Freedom House, and Varieties of Democracy. The measures developed by the Varieties of Democracy project outperform Polity2 and Freedom House Index with respect to the underlying definition and measurement scale, as well as the theoretical justification of the aggregation procedure. The three indices display a high level of agreement for those observations included in all three datasets. The most substantial differences between the indices lie in the indices’ coverage, i.e. in their non-missing observations (in Polity2 coding, for example, years during which a country is occupied by foreign powers constitute missing values), the availability of disaggregate data and the above mentioned key areas. This paper clarifies when to proceed with caution, but for the most part advocates the use of Varieties of Democracy in the statistical analysis of democracy.
Get full access to this article
View all access and purchase options for this article.
References
Acemoglu D, Naidu S, Restrepo P, et al. (2014) Democracy does cause growth. Technical report, National Bureau of Economic Research.
Boese VA, Kamin K (2018) ’tis but thy name that is my enemy: On the construction of macro panel datasets in peace economics. Economics of Peace and Security Journal (forthcoming).
Bogaards M (2010) Measures of democratization: From degree to type to war. Political Research Quarterly 63(2): 475–488.
Boix C, Miller M, Rosato S (2013) A complete data set of political regimes, 1800–2007. Comparative Political Studies 46(12): 1523–1554.
Bollen K (1993) Liberal democracy: Validity and method factors in cross-national measures. American Journal of Political Science 37(4): 1207–1230.
Bollen KA (1980) Issues in the comparative measurement of political democracy. American Sociological Review 45(3): 370–390.
Bollen KA, Jackman RW (1985) Political democracy and the size distribution of income. American Sociological Review 50(4): 438–457.
Bollen KA, Jackman RW (1989) Democracy, stability, and dichotomies. American Sociological Review 54(4): 612–621.
Bollen KA, Paxton P (2000) Subjective measures of liberal democracy. Comparative Political Studies 33(1): 58–86.
Bush SS (2017) The politics of rating freedom: Ideological affinity, private authority, and the freedom in the world ratings. Perspectives on Politics 15(3): 711–731.
Casper G, Tufis C (2003) Correlation versus interchangeability: The limited robustness of empirical findings on democracy using highly correlated data sets. Political Analysis 11(2): 196–203.
Cheibub JA, Gandhi J, Vreeland JR (2010) Democracy and dictatorship revisited. Public Choice 143(1–2): 67–101.
Coppedge M, Alvarez A, Maldonado C (2008) Two persistent dimensions of democracy: Contestation and inclusiveness. The Journal of Politics 70(3): 632–647.
Coppedge M, Gerring J, Altman D, et al. (2011) Conceptualizing and measuring democracy: A new approach. Perspectives on Politics 9(2): 247–267.
Coppedge M, Gerring J, Lindberg SI, et al. (2017a) V-dem comparisons and contrasts with other measurement projects. V-Dem Working Paper 2017:45. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2951014## (accessed 21 December 2017).
Coppedge M, Gerring J, Lindberg SI, et al. (2017b) V-dem codebook v7. V-Dem Reference Material (2017b). Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2968274 (accessed 21 December 2017).
Coppedge M, Gerring J, Lindberg SI, et al. (2017c) V-dem dataset v7. Available at: https://www.v-dem.net/en/data/data-version-7–1/.
Coppedge M, Gerring J, Lindberg SI, et al. (2017d) V-dem country coding units v7. V-Dem Reference Material, Version 7, May 2017. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2968285 (accessed on 21 December 2017).
Coppedge M, Gerring J, Lindberg SI, et al. (2017e) V-dem methodology v7. V-Dem Reference Material (2017). Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2968284 (accessed on 21 December 2017).
Coppedge M, Gerring J, Lindberg SI, et al. (2017f) V-dem organization and management v7. V-Dem Reference Material (2017). Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2968280 (accessed on 21 December 2017).
Crick B (2002) Democracy: A Very Short Introduction, volume 75. Oxford Paperbacks. Oxford: Oxford University Press.
Dahl RA (1971) Polyarchy: Participation and Opposition. New Haven and London: Yale University Press.
Diamond L (1999) Developing Democracy: Toward Consolidation. Baltimore, MA: Johns Hopkins University Press.
Eckstein H (1973) Authority patterns: A structural basis for political inquiry. American Political Science Review 67(4): 1142–1161.
Elff M, Ziaja S (2018) Method factors in democracy indicators. Politics and Governance 6(1): 92–104.
Elkins Z (2000) Gradations of democracy? Empirical tests of alternative conceptualizations. American Journal of Political Science 44(2): 293–300.
Fearon JD, Laitin DD (2003) Ethnicity, insurgency, and civil war. American Political Science Review 97(1): 75–90.
Franco Á, Álvarez-Dardet C, Ruiz MT (2004) Effect of democracy on health: Ecological study. British Medical Journal 329(7480): 1421–1423.
Gates S, Hegre H, Jones MP, et al. (2006) Institutional inconsistency and political instability: Polity duration, 1800–2000. American Journal of Political Science 50(4): 893–908.
Gleditsch KS, Ward MD (1997) Double take: A reexamination of democracy and autocracy in modern polities. Journal of Conflict Resolution 41(3): 361–383.
Goertz G (2006) Social Science Concepts: A User’s Guide. Princeton, NJ: Princeton University Press.
Goldstone JA, Bates RH, Epstein DL, et al. (2010) A global model for forecasting political instability. American Journal of Political Science 54(1): 190–208.
Gurr TR (1974) Persistence and change in political systems, 1800–1971. American Political Science Review 68(4): 1482–1504.
Hegre H (2001) Toward a democratic civil peace? Democracy, political change, and civil war, 1816–1992. American Political Science Review 95(1): 33–48.
Högström J (2013) Does the choice of democracy measure matter? Comparisons between the two leading democracy indices, Freedom House and Polity IV. Government and Opposition 48(2): 201–221.
House F (2017a) Freedom in the world 2016: Anxious dictators, wavering democracies: Global freedom under pressure. Available at: https://freedomhouse.org/sites/default/files/FH_FITW_Report_2016.pdf (accessed on Dec. 1, 2017) .
House F (2017b) Methodology website. Available at: https://freedomhouse.org/report/freedom-world-2016/methodology (accessed on Dec. 1, 2017).
Høyland B, Moene K, Willumsen F (2012) The tyranny of international index rankings. Journal of Development Economics 97(1): 1–14.
Huntington SP (1993) The Third Wave: Democratization in the Late Twentieth Century, volume 4. University of Oklahoma Press.
Inkeles A (1991) On Measuring Democracy: Its Consequences and Concomitants. New Brunswick, NJ: Transaction Publishers.
Knutsen CH (2010) Measuring effective democracy. International Political Science Review 31(2): 109–128.
Knutsen CH (2011) The economic effects of democracy and dictatorship. PhD Thesis, University of Oslo, Norway.
Lindberg SI, Coppedge M, Gerring J, et al. (2014) V-dem: A new way to measure democracy. Journal of Democracy 25(3): 159–169.
Mainwaring S, Brinks D, Pérez-Liñán A (2007) Classifying political regimes in Latin America, 1945–2004. In: Gerardo LM (ed) Regimes and Democracy in Latin America: Theories and Methods. Oxford: Oxford University Press, pp.123–160.
Marquardt KL, Pemstein D (2018) IRT models for expert-coded panel data. Political Analysis 26(4): 431–456.
Marshall MG, Gurr TR, Jaggers K (2017a) Polity IV project: Dataset users’ manual—political regime characteristics and transitions, 1800–2016. Available at http://www.systemicpeace.org/inscrdata.html (accessed 06 July 2017).
Marshall MG, Gurr TR, Jaggers K (2017b) Polity IV project: Political regime characteristics and transitions, 1800–2016. Available at http://www.systemicpeace.org/inscrdata.html (accessed 06 July 2017).
Munck GL (2009) Measuring Democracy: A Bridge Between Scholarship and Politics. Baltimore, MA: Johns Hopkins University Press.
Munck GL, Verkuilen J (2002) Conceptualizing and measuring democracy: Evaluating alternative indices. Comparative Political Studies 35(1): 5–34.
Pemstein D, Meserve SA, Melton J (2010) Democratic compromise: A latent variable analysis of ten measures of regime type. Political Analysis 18(4): 426–449.
Przeworski A, Alvarez M, Cheibub JA, et al. (2000) Democracy and Development: Political Institutions and Well-Being in the World, 1950–1990, volume 3. Cambridge, CA: Cambridge University Press.
Schmitter PC, Karl TL (1991) What democracy is… and is not. Journal of Democracy 2(3): 75–88.
Schumpeter JA (1976 [1942]) Capitalism, Socialism and Democracy. London and New York: Routledge.
Treier S, Jackman S (2008) Democracy as a latent variable. American Journal of Political Science 52(1): 201–217.
Vanhanen T (2000) A new dataset for measuring democracy, 1810–1998. Journal of Peace Research 37(2): 251–265.
Vreeland JR (2008) The effect of political regime on civil war: Unpacking anocracy. Journal of Conflict Resolution 52(3): 401–425.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
- ias815571_appendix.pdf
- 645.12 KB
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: May 29, 2019
Issue published: June 2019
Keywords
Authors
Metrics and citations
Metrics
Journals metrics
This article was published in International Area Studies Review.
View All Journal MetricsArticle usage*
Total views and downloads: 9499
*Article usage tracking started in December 2016
Altmetric
See the impact this article is making through the number of times it’s been read, and the Altmetric Score.
Learn more about the Altmetric Scores
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 70 view articles Opens in new tab
Crossref: 78
- Does sovereignty help economic growth? A recent reassessment
- The Effect of Political Regime on the Association of Values with Subje...
- Three-Dimensional Corruption Metrics: A Proposal for Integrating Frequ...
- Regime Type and Data Manipulation: Evidence from the COVID-19 Pandemic
- Rasch model and anchoring vignettes for metrological characterisation ...
- ON THE RELATIONSHIP BETWEEN DEMOCRACY AND SAFETY (RESULTS INDICATED BY...
- In-group we trust, no more? How senders’ legitimacy shapes the success...
- Democracy, Egalitarianism and the Homicide Rate: An Empirical Test of ...
- International Monetary Fund Conditionalities May Weaken the Recipient ...
- You have nothing to lose but your chains?
- Towards environmental impact of inward foreign direct investment: the ...
- Interpretative structural modeling to social sciences: designing bette...
- Climate policy uncertainty and environmental degradation: Does democra...
- Populism and authoritarianism in the EU: what is the role of the euro?
- Toward sustainable energy transition: Unveiling the synergies of democ...
- A new dynamic framework is required to assess adaptation limits
- Measuring Political Institutions in the Long Run: A Latent Variable An...
- Democracy, accountability, and forest loss: a cross-national analysis ...
- Corruption and Palm Oil in a Cross-National Perspective: How India Con...
- Freedom House ve Metodolojisi Üzerine Eleştirel Bir Değerlendirme: Dem...
- Are democratic regime and the magnitude of the informal economy robust...
- The political economy of inequality, mobility and redistribution
- The legal foundation of responsible research assessment: An overview o...
- Policy process theories in autocracies: Key observations, explanatory ...
- Difficult to Count, Important to Measure: Assessing Democratic Backsli...
- Social media, quality of democracy, and citizen satisfaction with demo...
- Reanalysing the link between democracy and economic development
- Backsliding among indicators of democratic stability relevant to publi...
- Government revenue composition and forest loss: A cross‐national study...
- Democracy, Peace, and Religion in Nigeria: Can Religion Be Used to Con...
- Connecting Bureaucratic Structures to Forest Loss
- Social media, disinformation, and democracy: how different types of so...
- Panel data on the impact of democracy and political stability on econo...
- Threat or corrective to democracy? The relationship between populism a...
- Political Accountability and Military Spending
- Episodes of liberalization in autocracies: a new approach to quantitat...
- Civil War Outcomes and Egalitarian Democracy in Post-Civil War States:...
- Death by a thousand cuts: measuring autocratic legalism in the Europea...
- Clientelism and Forest loss in a Macro-Comparative perspective
- Anti-democratic youth? The influence of youth cohort size and quality ...
- Praising the leader: personalist legitimation strategies and the deter...
- From the Varieties of Democracy to the defense of liberal democracy: V...
- Unravelling the link between democracy and economic complexity: fresh ...
- Measuring Partial Democracies: Rules and their Implementation
- UN Peacekeeping and Democratization in Conflict-Affected Countries
- You Have Nothing to Lose but Your Chains? Re-examining the Hayek-Fried...
- La calidad de la democracia en América: un análisis comparado, 1972-20...
- The power of a promise: whom do governments’ security justifications c...
- Measuring democracy among ordinary citizens—Challenges to studying dem...
- Patterns of Democracy over Space and Time
- An unexpected consensus among diverse ways to measure democracy
- Can democracy aid improve democracy? The European Union’s democracy as...
- Financial System Development and Democracy: a Panel Smooth Transition ...
- Democracy and terrorism in Africa
- Youth Cohort Size, Structural Socioeconomic Conditions, and Youth Prot...
- A century of coalitions in battle: Incidence, composition, and perform...
- Should we care (more) about data aggregation?
- Corrupting Ecologically Unequal Exchange? India and Forest Loss in a C...
- Democracy Measurement Redux: A Critical Look at Some New Kids on the B...
- A Short History of Contestation and Participation
- Tariff Revenues Matter for Democratization: Theory and Evidence From t...
- Using Machine Learning for measuring democracy: A practitioners guide ...
- Comparing measures of democracy: statistical properties, convergence, ...
- A business case for democracy: regime type, growth, and growth volatil...
- Political liberalization and human development: Dynamic effects of pol...
- The politics of academic innovation: A cross-national study of the eff...
- Conceptualizing and Measuring Autocratization Episodes
- Quality of Democracy Makes a Difference, but Not for Everyone: How Pol...
- Measuring Governance to Achieve Sustainable Development: Promises and ...
- Marketplace of indicators: inconsistencies between country trends of m...
- Using Machine Learning for Measuring Democracy: An Update
- Democracy and Terrorism in Africa
- Measuring Governance to Achieve Sustainable Development: Promises and ...
- New evidence on the oil-democracy nexus utilising the Varieties of Dem...
- Comparing public communication in democracies and autocracies: automat...
- Linking democracy and biodiversity conservation: Empirical evidence an...
- Democratic Legacies: Using Democratic Stock to Assess Norms, Growth, a...
- What Have We Learned About Political Participation Changing a Financia...
Figures and tables
Figures & Media
Tables
Get access
Get access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
Access journal content via a university, library or employer subscription.
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
