FIND DATA: By Author | Journal | Sites   ANALYZE DATA: Help with R | SPSS | Stata | Excel   WHAT'S NEW? US Politics | Int'l Relations | Law & Courts
   FIND DATA: By Author | Journal | Sites   WHAT'S NEW? US Politics | IR | Law & Courts
If this link is broken, please report as broken. You can also submit updates (will be reviewed).
When List Experiments Mislead: Fixing Measurement Error in Sensitive Surveys
Insights from the Field
list experiment
measurement error
NLSreg
MLreg
misspecification test
Methodology
Pol. An.
1 archives
Dataverse
List Experiments With Measurement Error was authored by Graeme Blair, Winston Chou and Kosuke Imai. It was published by Cambridge in Pol. An. in 2019.

🔍 Problem Identified:

Measurement error undermines survey validity, particularly for sensitive questions. List experiments reduce deliberate misreporting but remain vulnerable to nonstrategic measurement error from poor implementation and respondent inattention. Such error violates the assumptions of the standard maximum likelihood regression (MLreg) estimator and can produce misleading inferences—especially when the sensitive trait is rare.

📊 How the paper evaluates robustness and detection:

  • Shows that the nonlinear least squares regression (NLSreg) estimator introduced in Imai (2011) is robust to nonstrategic measurement error.
  • Proposes a general model misspecification test that flags divergence between MLreg and NLSreg estimates as a diagnostic for measurement error.

🛠️ How measurement error is addressed directly:

  • Introduces new estimators that explicitly model nonstrategic measurement error.
  • These estimators retain the statistical efficiency advantages of MLreg while improving robustness to the kinds of implementation and attention failures that bias standard estimates.

📁 Empirical checks and practical tools:

  • Reanalyzes published list-experiment studies previously found to exhibit nonstrategic measurement error and demonstrates that the proposed diagnostics readily detect bias and that the new estimators mitigate it.
  • Offers a set of practical recommendations for applied researchers on diagnosing and correcting measurement error in list experiments.

💻 Implementation:

  • All proposed methods are implemented in an open-source software package to facilitate adoption and replication.

Why it matters: The suite of diagnostic tests and estimation tools allows researchers to detect when list experiments are compromised by inattentive or flawed implementation, to choose estimators that remain valid in those settings, and to recover more reliable estimates—critical when the target trait is rare and the risk of bias is greatest.

data
Find on Google Scholar
Find on JSTOR
Find on CUP
Political Analysis
Podcast host Ryan