How the Presence of Others
Can Bias Survey Responses

 

Gustavo Diaz
McMaster University

Michelle Dion
McMaster University

Guillem Riambau
Universitat de Barcelona Institut d’Economia de Barcelona

 

Slides and paper: talks.gustavodiaz.org

Disclaimer

Data harmonization still in progress!

Sensitive survey questions

  1. A social referent…

  2. … who can learn the response

  3. … has a preferred response

  4. Cost of wrong response

 

Presence of others meets all criteria

What we know so far

  • Presence of enumerators: Political questions are sensitive

  • Household surveys: Presence of parents/spouses shapes answers

  • Afrobarometer: Family and non-family affect responses and don’t know rates (Zimbalist 2022)

This project

  1. More data and more details

  2. Determine severity of problem

This project

  1. More data and more details

  2. Determine severity of problem

Asian Barometer Surveys

  • Waves 1-5 (2000-2021)

  • 1,000-1,800 respondents per country-wave

  • 15 countries, 21 years, 97,439 interviews

  • 95% Detailed record of who was present

  • 45% of interviews had others present

  • Partner/spouse > Children > Parents/in-laws > Nonfamily > Party/gov officials

Coverage

Sensitive questions

  1. Country is a democracy

  2. Satisfaction with democracy

  3. System of government is capable of solving country’s problems

  4. I would rather live under our system of government

  5. Government will solve most important problem

Results: Refusal to answer

Results: Agreement to questions

Next steps

  • Questions about trust, perceptions of corruption

  • Determine whether presence of others biases statistical inferences

  • Takeaway: Take into account in research design and analysis