Getting to the bottom of response behavior when using Forced Answering in Online Surveys

Abstract:
Recent studies have shown that the use of the forced answering (FA) option in online surveys results in reduced data quality. This response behavior has often been interpreted as psychological reactance reaction. However, no study researched the psychological mechanism behind the correlation of FA on dropout and data quality before. By using online survey-experiments with forced and non-forced answering instructions, our study offers statistical evidence for the often proposed reactance effect influencing response behavior.

Relevance:
Recent studies have shown that the use of the forced answering (FA) option in online surveys results in reduced data quality. They especially examined that forcing respondents to answer questions in order to proceed through the questionnaire leads to higher dropout rates and lower answer quality. However, no study researched the psychological mechanism behind the correlation of FA on dropout and data quality before. This response behavior has often been interpreted as psychological reactance reaction. So, the Psychological Reactance Theory (PRT) predicts that reactance appears when an individuals’ freedom is threatened and cannot be directly restored. Reactance describes the motivation to restore this loss of freedom. Respondents could experience FA as a loss of freedom, as (s)he is denied the choice to leave a question unanswered. According to PRT, possible reactions in this situation might be to quit survey participation, to fake answers or to show satisficing tendencies.
Research content:
This study explores the psychological mechanism that effects response behavior in FA condition (compared to non-FA- condition). Our major hypothesis is that forcing respondents to answer will cause reactance, which turns into increasing dropout rates, decreasing answer quality and a satisficing behavior.
Methods and Data:
We used online survey-experiments with forced and non-forced answering instructions. Reactance was measured with a four-item reactance scale. To determine answer quality, we used self-report for faking as well as the analysis of answers to open ended questions.
Results:
Zero-order effects showed that FA increased state reactance and questionnaire dropout as well as it reduced answer length in open-ended questions. Mediation analysis supported the hypothesis of reactance as an underlying psychological mechanism behind negative FA effects on data quality.
Added Value:
This is the first study which offers statistical evidence for the often proposed reactance effect influencing response behavior. This offers a base for a deeper psychological reflection of the use of the FA-option.