--> Hereafter some interesting papers...
Moving Off- to On-line When a research study is moved from an offline methodology to online, results will change and, because they often change in unpredictable ways, weighting the online results to match old data can be a waste of time and effort.
HERE ARE SOME INSIGHTS
based on hundreds of projects moving from offline to online:
Biases exist with every methodology and when differences are seen after moving a study online, there are three likely sources:
Sample effects:
- Non-coverage of cell phone households with telephone methodology or of non-internet with online methodology.
- Non-response of people who won’t take phone surveys with telephone methodology or of people who won’t join panels with online methodology.
Mode effects:
- Interviewer: With no interviewer present online, respondents may be more honest and less likely to give socially desirable responses, but may also be more likely to quit the survey without interviewer encouragement.
- Visual v. aural: There is more use of end points on a scale when it is read to the respondent; more use of midpoints online when the entire scale is constantly visible.
- Access: Less affluent respondents may have slower internet connections, making their survey experience different from that of other groups.
- Visual cues: when switching from face to face to online, the visual cues in a home, which can provide additional data (e.g. product ownership) are not available.
Questionnaire Effects:
- Use of graphics, variation in question wording and pacing can cause differences (online the speed of the questionnaire is dictated by respondent, while offline the interviewer controls it).
POINT OF VIEW
When moving projects online:
- Facts reported by respondents are unlikely to change; reported opinions and attitudes may change.
- Spontaneous awareness measures are likely to increase since online respondents can take longer to think without feeling pressured to give an answer or move on with the survey. Prompted awareness measures are unlikely to change
- There will be more use of midpoints, and more use of “don’t know” (less tendency to “agree” with the interviewer or invent something in order to appear “good” in the interviewer’s eyes.)
- The “social desirability” differences will vary in direction and strength depending on culture. In countries where internet penetration is low, sampling effects due to coverage are more significant.
RECOMMENDATIONS
When moving projects online:
- Run projects side by side for several iterations to compare results and help understand differences • Redesign the survey instrument. Don’t put a phone script up online.
- Always run a test of the new survey before launching the new questionnaire
- Expect differences, observe, document and understand them instead of weighting to old methodologies.
