An Updated Protocol to Detect Invalid Entries in an Online Survey of Men Who Have Sex with Men (MSM): How Do Valid and Invalid Submissions Compare?

• Type of Material: Article
• Authors: Jeremy Grey, Emory University; Joseph Konstan, University of Minnesota; Alex Iantaffi, University of Minnesota; J. Michael Wilkerson, University of Texas; Dylan Galos, University of Minnesota; B.R. Simon Rosser, University of Minnesota
• Language: English
• Date: 2015-10
• Publisher: Springer Verlag (Germany)
• Publication Version: Author Accepted Manuscript (After Peer Review)
• Copyright Statement: © 2015, Springer Science+Business Media New York.
• Final Published Version (URL): http://dx.doi.org/10.1007/s10461-015-1033-y
• Title of Journal or Parent Work: AIDS and Behavior
• ISSN: 1090-7165
• Volume: 19
• Issue: 10
• Start Page: 1928
• End Page: 1937
• Grant/Funding Information: The Sexually Explicit Media (SEM) Study was funded by the National Institute of Mental Health (NIMH), Grant #5R01MH087231-02.
• Keywords: Internet; Questionnaires; HIV Infections; Data Collection; Validity; Adult; Reproducibility of Results; Surveys and Questionnaires; Male; Bias (Epidemiology); Survey methods; HIV; Data Accuracy; Bias; Homosexuality, Male; Humans
• Research Categories: Health Sciences, Epidemiology; Psychology, Behavioral; Health Sciences, Public Health

Abstract

Researchers use protocols to screen for suspicious survey submissions in online studies. We evaluated how well a de-duplication and cross-validation process detected invalid entries. Data were from the Sexually Explicit Media Study, an Internet-based HIV prevention survey of men who have sex with men. Using our protocol, 146 (11.6 %) of 1254 entries were identified as invalid. Most indicated changes to the screening questionnaire to gain entry (n = 109, 74.7 %), matched other submissions’ payment profiles (n = 56, 41.8 %), or featured an IP address that was recorded previously (n = 43, 29.5 %). We found few demographic or behavioral differences between valid and invalid samples, however. Invalid submissions had lower odds of reporting HIV testing in the past year (OR 0.63), and higher odds of requesting no payment compared to check payments (OR 2.75). Thus, rates of HIV testing would have been underestimated if invalid submissions had not been removed, and payment may not be the only incentive for invalid participation.