Why Conjoint Results Depend on Profile Distribution

🔍 What This Paper Shows

Conjoint analysis measures multidimensional preferences by estimating the average marginal component effect (AMCE) — the causal effect of a single profile attribute averaged over the other attributes. The AMCE, however, depends critically on the distribution used for that averaging. Most experiments default to a uniform distribution that weights every profile equally, but real-world profile frequencies and theoretically relevant counterfactual distributions are often far from uniform. This mismatch can seriously undermine the external validity of conjoint findings.

🧪 How the Argument Is Demonstrated

• Documents that common practice uses uniform profile weighting while target distributions often differ.
• Empirically compares AMCE estimates computed under uniform averaging versus averaging over target profile distributions.
• Uses two empirical applications: one that applies a real-world profile distribution and another that uses a counterfactual distribution motivated by a theoretical question.

📈 Key Findings

• AMCE estimates can change substantially when averaging over a target profile distribution instead of the uniform distribution.
• The direction and magnitude of these changes depend on how the other attributes are distributed in the target population or theoretical scenario.

🔧 Practical Fixes and Tools

• Proposes new experimental designs and estimation methods that explicitly incorporate substantive knowledge about the profile distribution into both design and analysis.
• Demonstrates how these approaches improve the external validity of conjoint estimates for both real-world and counterfactual questions.
• Implements the proposed methodology in an open-source software package to facilitate adoption.

📌 Why It Matters

Researchers and policymakers using conjoint analysis should account for the target profile distribution when estimating AMCEs. Failing to do so risks drawing conclusions that do not generalize to the contexts of substantive interest.