Jelke Bethlehem - Handbook of Web Surveys

3 Exercise 2.3 Which of the following phenomena is not a problem of self‐selection web surveys?A respondent can complete a questionnaire more than once.Persons not belonging to the target population can complete the questionnaire.The survey results show a lack of representativity.It is difficult to get a large number of respondents.

4 Exercise 2.4 What is the difference between a cross‐sectional survey and a longitudinal survey?A cross‐sectional survey measures changes over time, and a longitudinal survey measures the state of the population at one point in time.A cross‐sectional survey measures the state of a population at one moment in time, and a longitudinal survey measures time changes over time.A cross‐sectional survey mainly measures facts and behavior, and a longitudinal survey measures attitudes and opinions.For cross‐sectional surveys, any mode of data collection is adequate, whereas in longitudinal surveys only Internet is possible.

5 Exercise 2.5 What is offline data collection?Any form of data collection that does not use the Internet.A form of Internet data collection for which the questionnaire is not written in HTML.A survey that uses e‐mail to transfer information.A survey that uses the Internet to transfer the electronic questionnaire to the respondents.

6 Exercise 2.6 What is the main reason national statistical institutes consider using web surveys?It shows government also uses modern ICT.It reduces nonresponse in surveys.It improves the quality of the collected data.It reduces data collection costs.

1 Ajzen, I. (1991), The Theory of Planned Behavior. Organizational Behavior and Human Decision Processes, 50, pp. 179–211.

2 Biffignandi, S. (2010a), Modeling Non‐sampling Errors and Participation in Web Surveys. Proceedings of the 45th SIS Scientific Meeting, Padova, Italy.

3 Biffignandi, S. (2010b), Internet Survey Methodology—Recent Trends and Developments. In: Lovric, M. (ed.), International Encyclopedia of Statistical Science. Springer, Heidelberg, Germany.

4 Biffignandi, S. & Pratesi, M. (2002), Internet Surveys: The Role of Time in Italian Firms Response Behaviour. Research in Official Statistics, 5, pp. 19–33.

5 Brown, J. A., Serrato, C. A., Hugh, M., Kanter, M. H., Spritzer, K. L., & Hays, R. D. (2016), Effect of a Post‐paid Incentive on Response Rates to a Web‐Based Survey. Survey Practice, 9, 1. https://doi.org/10.29115/SP‐2016‐0001.

6 Crawford, S. D., Couper, M. P., & Lamias, M. J. (2001), Web Surveys. Perceptions of Burden. Social Science Computer Review, 19, pp. 146–162.

7 Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014), Internet, Mail and Mixed‐Mode Surveys. The Tailored Design Methods. 4th Edition, Wiley, Hoboken, NJ.

8 DiSogra, C. & Callegaro, M. (2009), Computing Response Rates for Probability‐Based Web Panels. Proceeding of Section on Survey Research of the American Statistical Association, Washington, DC.

9 Göritz, A. (2006), Incentives in Web Studies: Methodological Issues and a Review. International Journal of Internet Science, 1, pp. 58–70.

10 Göritz, A. (2010), Using Lotteries, Loyalty Points, and Other Incentives to Increase Participant Response and Completion. In: Gosling, S. & Johnson, J. (eds.), Advanced Methods for Conducting Online Behavioral Research, pp. 219–233. American Psychological Association, Washington, DC. doi: https://doi.org/10.1037/12076‐014.

11 Göritz, A. (2015), Incentive Effects. In: Engel, U., Jann, B., Lynn, P., Scherpenzeel, A., & Sturgis, P. (eds.), Improving Survey Methods: Lessons from Recent Research, pp. 339–350. Routledge, London.

12 Lozar Manfreda, K. & Vehovar, V. (2008), Internet Surveys. In: de Leeuw, E., Hox, J. J., & Dillman, D. A. (eds.), International Handbook of Survey Methodology. Lawrence Erlbaum Associates, New York.

13 Singer, E. & Ye, C. (2013), The Use of Incentives in Surveys. Annals of the American Academy of Political and Social Sciences, 645 (1), pp. 112–141.

14 Steinmetz, S., Bianchi, A., Tijdens, K., & Biffignandi, S. (2014), Improving Web Survey Quality—Potentials and Constraints of Propensity Score Weighting. In: Callegaro, M., Baker, R., Bethlehem, J., Göritz, A., Krosnick, J.A., & Lavrakas, P. J. (eds.), Online Panel Research: A Data Quality Perspective. Wiley, Chichester. pp. 273–298.

The framework proposed in this chapter is a collection of interrelated ideas and concepts that provide guidance for researchers undertaking a web survey. To carry out a survey, one must conduct a process that involves many steps and decisions. The process‐oriented approach divides the survey into steps that define the flow of the survey process, and this approach must be applied to web surveys as well. Steps differ in some respects from the traditional modes. Thus, the flow of the web survey process must be explicitly defined. Bethlehem and Biffignandi (2012) and Tourangeau, Conrad, and Couper (2013) pointed out the specifics of the web procedures without focusing on the whole process. Thorsdottir and Biffignandi 1 presented a flowchart for the steps of a probability‐based web survey. This chapter follows the flowchart approach to illustrate the process, the decisions involved at the various steps of the process, and the related risks of errors. The flowchart will help the surveyor 2 in properly planning a web survey and will help researchers highlight the steps of the survey process for their respective research. Starting from the flowchart presented by Thorsdottir and Biffignandi (Webdatanet Conference, 2015), the flowchart presented in this chapter is slightly different since mobile extension of the web survey is also considered (i.e., mobile web survey). Considering the wide diffusion of mobile devices (especially smartphones), mobile web survey is becoming a very frequent, say, mostly a standard, situation. When a web survey is administered if no disallow is activated, the survey may be received and completed not only using a PC but even from a mobile device. This situation is called mobile web survey, i.e., no prevention toward mobile devices in a web survey (for a discussion of the mobile surveys, see Chapter 2). For a detailed description of the choices for smartphone participation in web surveys, see Peterson et al. (2017). Mobile web survey implies several methodological challenges; nevertheless the survey process requires only a few differences with respect to the web‐only survey.

The flowchart divides the mobile web (or the web‐only survey) process into the following six major steps:

1 Determine the survey objective.

2 Metadata definition.

3 Designing the mobile web or only web survey (deciding on the mode, the sampling frame, and the sampling approach, designing the questionnaire, designing the paradata methodology, and selecting the sample, software, or programming language).

4 Collecting the data.

5 Creating the database.

6 Processing data.

The use of the flowchart brings a total survey error (TSE) perspective, which is an effective approach for understanding error sources in a comprehensive way (Weisberg, 2005). Biemer TSE approach aims at helping researchers make design decisions that maximize data quality within the constraints of a limited budget. TSE perspective disaggregates overall error into component, mainly distinguishing between sampling and non‐sampling errors. The TSE paradigm (see, for example, Platek and Särndal, 2001 and the ensuing discussions) refers to the concept of optimally allocating the available survey resources to minimize TSE for key estimates. In principle, to apply the TSE paradigm, the major sources of error should be identified so that the survey resources can be allocated to reduce their errors, within specified constraints on costs. An overview of the TSE history and recent research results on the relationships between different types of errors is given in Biemer et al. (2017). The idea of the TSE is of optimally balancing the dimensions of survey quality within the survey budget and schedule. It is possible to distinguish between TSE and total survey quality. This considers the fitness for use of an estimate, that is, quality from both the producer and user perspectives. The “fitness for use” concept implies not only accuracy is considered; for instance, attributes such as the timeliness, accessibility, and comparability of the data have to be granted in the survey process, as well as various types of errors (see Chapter 5), such as measurement, nonresponse, and so forth (Groves, 1989).