Careless Responding in Online Survey Research

Much of our recent research on relational aggression has utilized college student samples and has involved online surveys. Based on published recommendations (e.g., Huang, Curran, Keeney, Poposki, & DeShon, 2011; Liu, Bowling, Huang, & Kent, 2013; Meade & Craig, 2012), we have been incorporating various methods of detecting careless responding in our surveys. What we have found is that a substantial number of research participants are responding carelessly. In the interest of data integrity, it is clear that the use of procedures to detect careless responders are essential to include in online survey research.

For those researchers just beginning to consider incorporating methods for identifying careless responders and reducing careless responding in online survey research, some of the procedures we have been using include:
  • Modifying consent forms and survey instructions to inform potential participants that quality assurance checks are being used and that failing such checks will result in them not receiving incentives for participation
  • Including validity items or bogus items that should be answered the same way by participants who are attending to item content
  • Measuring survey completion and/or individual instrument completion time
The use of these procedures has allowed us to make sure that participants who are responding carelessly do not receive incentives for participation (e.g., research credit) and that we can easily identify and remove their data.

We have noticed that it is becoming increasingly common for authors of studies using online surveys to address how they detected careless responders and what they did with these data. This suggests that the use of such procedures are rapidly becoming part of routine practice to promote data integrity.