Abstract: How well do vignette designs capture actual behaviour in the real world? This study employs original survey data featuring both hypothetical vignettes and behavioural questions in order to assess the external validity of descriptive and causal inferences in survey experiments. The survey was conducted in a three-province, probability-proportional-to-size sample of 1,897 rural residents in China and focuses on the legal mobilization of citizens in response to grievances involving land rights. In terms of descriptive inference, we find that relative to the behavioural benchmark, hypothetical vignettes significantly over-estimate legal mobilization in response to a grievance, particularly for higher-cost actions like petitioning the government and litigating in court. We find that data from hypothetical vignettes affect causal inference as well, producing significantly different results regarding the effect of political connections and legal knowledge on legal mobilization. The study makes a contribution by identifying conditions under which hypothetical vignettes are less likely to produce valid inference. It engages a rich literature on disputing and legal mobilization in the field of Chinese politics and helps to resolve debates over the role of political connections and legal knowledge.
摘要: 问卷调查中的情境设计多大程度上能反映现实世界中的行为？本研究采用一个同时包含了虚拟情境和真实行为问题的原创性问卷，分析情境设计在获得描述统计和因果推论分析中的外部效度。该问卷采用按规模大小成比例的概率抽样法，在中国的三个省份调查了1897 名农村居民，问题主要涵盖了农村居民在涉及土地权利纠纷时的法律动员状况。我们发现，相比在现实中遭遇土地纠纷的受访者，那些在情境题中被问及会如何回应虚拟的土地纠纷的受访者有更高的比例愿意采用法律手段解决纠纷，特别是那些施行成本较高的手段，比如上访或起诉。除了描述统计上的差异，我们发现情境设计同样影响了因果推论分析，例如政治关系和法律知识是否对采取法律手段产生影响。本研究的主要贡献在于阐明了情境设计在何种条件下使用更为有效。此外本文也与中国政治领域中关于纠纷和法律动员的丰富文献进行对话，并尝试回应文献中围绕政治关系和法律知识的作用的讨论。