FIND DATA: By Author | Journal | Sites   ANALYZE DATA: Help with R | SPSS | Stata | Excel   WHAT'S NEW? US Politics | Int'l Relations | Law & Courts
   FIND DATA: By Author | Journal | Sites   WHAT'S NEW? US Politics | IR | Law & Courts
If this link is broken, please report as broken. You can also submit updates (will be reviewed).
Insights from the Field

Surprising Accuracy in Measuring Democracy Without Coder Bias


observables
random forest
measurement bias
electoral democracy
Comparative Politics
AJPS
6 Datasets
1 PDF
1 Text
40 Other
Dataverse
Measuring Electoral Democracy With Observables was authored by Daniel Weitzel, John Gerring, Daniel Pemstein and Svend-Erik Skaaning. It was published by Wiley in AJPS in 2025.

This article presents a novel approach to measuring electoral democracy using observable indicators. Traditional cross-national democracy indices rely heavily on subjective coder judgments that introduce potential biases and errors while being costly and time-consuming to produce. The authors address these limitations by gathering extensive data on various observable indicators (X') capturing different aspects of the democratic process, then applying supervised random forest machine learning to predict scores from these observables alone, creating an Observable-to-Subjective Score Mapping (OSM). The resulting index, Z', demonstrates minimal information loss compared with subjective measures.

Key Findings:

  • New method uses observable indicators rather than coder judgments
  • Machine learning algorithm produces accurate democracy scores
  • Reduces subjectivity and potential biases in measurement

Data & Methods:

  • Gathered data on diverse observable indicators (X')
  • Applied supervised random forest machine learning technique
  • Created Observable-to-subjective Score Mapping (OSM)

Why It Matters:

  • Provides cheaper, more accessible alternative to existing indices
  • Eliminates costly expert coding requirements
  • Offers potentially more reliable measurements across countries

Policy Implications:

  • Improved ability to track global democratic trends over time
  • More objective comparisons between different electoral systems
  • Potential for consistent democracy metrics in comparative research

The approach offers several advantages. It eliminates coder errors stemming from misinformation, slackness, or ideological biases toward specific regimes. The methodology is transparent and replicable with lower implementation costs than traditional expert coding approaches. This method potentially enables coverage of all polities worldwide—a significant advantage over existing indices that often rely on limited samples.

data
Find on Google Scholar
Find on JSTOR
Find on Wiley
Podcast host Ryan