Careless Responding in Online Survey Research
11/13/14 Research Methods
Much of our recent research on relational aggression has utilized college student samples and has involved online surveys. Based on published recommendations (e.g., Huang, Curran, Keeney, Poposki, & DeShon, 2011; Liu, Bowling, Huang, & Kent, 2013; Meade & Craig, 2012), we have been incorporating various methods of detecting careless responding in our surveys. What we have found is that a substantial number of research participants are responding carelessly. In the interest of data integrity, it is clear that the use of procedures to detect careless responders are essential to include in online survey research.
For those researchers just beginning to consider incorporating methods for identifying careless responders and reducing careless responding in online survey research, some of the procedures we have been using include:
We have noticed that it is becoming increasingly common for authors of studies using online surveys to address how they detected careless responders and what they did with these data. This suggests that the use of such procedures are rapidly becoming part of routine practice to promote data integrity.
For those researchers just beginning to consider incorporating methods for identifying careless responders and reducing careless responding in online survey research, some of the procedures we have been using include:
- Modifying consent forms and survey instructions to inform potential participants that quality assurance checks are being used and that failing such checks will result in them not receiving incentives for participation
- Including validity items or bogus items that should be answered the same way by participants who are attending to item content
- Measuring survey completion and/or individual instrument completion time
We have noticed that it is becoming increasingly common for authors of studies using online surveys to address how they detected careless responders and what they did with these data. This suggests that the use of such procedures are rapidly becoming part of routine practice to promote data integrity.