Jelke Bethlehem - Handbook of Web Surveys

Roos and Wings (2000) conducted a test with Internet data collection at Statistics Netherlands for the construction industry. Respondents could choose between three modes:

Completing a form offline. The form was sent as an HTML file that was attached to an e‐mail. The form is downloaded, completed offline, and returned by e‐mail.

Completing a form online. The Internet address of an online web form was sent by e‐mail. The form was completed online.

Completing an e‐mail form. An e‐mail is sent containing the questionnaire in plain text. Respondents clicked the reply button, answered the questions, and sent the e‐mail back.

A sample of 1,500 companies was invited to participate in the experiment. 188 companies were willing and able to participate. Of those, 149 could surf the Internet, and 39 only had e‐mail. Questionnaire completion times of all three modes were similar to that of a paper form. Respondents preferred the form‐based layout over the question‐by‐question layout. The conclusion of the experiment was that web surveys worked well.

General population web surveys were rare in the first period of existence of the Internet. This was due to the low Internet penetration among households. This prevented conducting representative surveys. However, there were polls on the Internet. Recruitment of respondents was based on self‐selection and not on probability sampling. Users could even create their own polls on websites like Survey Central , Open Debate, and Internet Voice (see O'Connell, 1998).

Also in 1998, the Survey2000 project was carried out. This was a large self‐selection web survey on the website of the National Geographic Society. This was a survey on mobility, community, and cultural identity. In a period of two months, over 80,000 respondents completed the questionnaire. See Witte, Amoroso, and Howard (2000) for more details about this project.

It seems to be typical for this type of self‐selection web surveys that they make it possible to collect data about a large number of respondents in a relatively short time. Other examples are given by Bethlehem and Stoop (2007). The survey 21minuten.nl has been conducted a number of times in the Netherlands. This survey supposed to supply answers to questions about important problems in Dutch society. Within a period of six weeks in 2006, about 170,000 people completed the online questionnaires. A similar survey was conducted in Germany. It is called Perspektive Deutschland . More than 600,000 participated in this survey in 2005/2006.

It should be noted that these large sample sizes are no guarantee for proper statistical inference. Due to under‐coverage (not everyone has access to the Internet) and self‐selection (no proper random sampling), estimates can be biased. This bias is independent of the sample size.

Internet penetration is still low in many countries, making it almost impossible to conduct a general population web survey. Since data collection costs can be reduced if the Internet is used, other approaches are sought. One such approach is mixed‐mode data collection . A web survey is combined with one or more other modes of data collection, like a mail survey, a telephone survey, or a face‐to‐face survey. Researchers first attempt to collect as much data as possible with the cheapest mode of data collection (web). Then, the nonrespondents are re‐approached in a different (next cheapest) mode. Example 1.6describes a survey run using a mixed‐mode approach.

Beukenhorst and Wetzels (2009) describe a mixed‐mode experiment conducted by Statistics Netherlands. They used the Dutch Safety Monitor for this experiment. This survey asks questions about feelings of security, quality of life, and level of crime experienced. The sample for this survey was selected from the Dutch population register. All sampled persons received a letter in which they were asked to complete the survey questionnaire on the Internet. The letter also included a postcard that could be used to request a paper questionnaire. Two reminders were sent to those that did not respond by web or mail. If still no response was obtained, nonrespondents were approached by means of CATI, if a listed telephone number was available. If not, these nonrespondents were approached by CAPI.

To be able to compare this four‐mode survey with a traditional survey, also a two‐mode survey was conducted for an independent sample. Sampled persons were approached by CATI if their telephone number was listed in the directory, and otherwise they were approached by CAPI.

The response rate for four‐mode survey turned out to be 59.7%. The response rate for the two‐mode survey was higher. So, introducing more modes did not increase the overall response rate. However, more than half of the response (58%) in the four‐mode survey was obtained with a self‐administered mode of data collection (web or paper). Therefore, the costs of the survey were much lower. Interviewers were deployed in only 42% of the cases. For more detail, see Beukenhorst and Wetzels (2009) or Bethlehem, Cobben, and Schouten (2011).

A special case of mixed‐mode data collection is related to the increasing diffusion of mobile phones and smartphones. When an invitation e‐mail is sent, the questionnaire might be received either on a computer or a mobile phone or a smartphone. The interviewee could complete the web questionnaire using either the mobile device or the computer. Thus, it is better to talk about mobile web surveys rather than web survey. A recommendation is to run web surveys that are fully adapt for smartphones. Therefore, in presenting methods for web surveys, comments about the adaption for smartphone surveys will be discussed all along the chapters of this handbook. Some penetration data allow for understanding how the situation differs across the countries. The coverage of telephone directories, of Internet, and of mobile cells provides the feeling of the need to adopt a mixed‐mode approach. A World Bank study reports that, in the 2018, Euro area fixed telephone subscription for 100 people is 44.4, mobile 122.6 and Internet 83.8. Table 1.2shows the same indicators by country. Only some countries are shown in the table since the objective is just to evidence that there is a relevant difference across countries.

Table 1.2 Penetration of fixed and mobile phone and of Internet (year 2018)

Source: Data from International Telecommunication Union. World Telecommunication/ICT.

Note that Internet users are individuals who have used the Internet (from any location) in the last 12 months. Internet can be used via a computer, mobile phone, personal digital assistant, games machine, digital TV, etc. Fixed telephone subscriptions refer to the sum of active number of analogue fixed telephone lines, voice‐over‐IP (VoIP) subscriptions, fixed wireless local loop (WLL) subscriptions, ISDN voice‐channel equivalents, and fixed public payphones. Mobile cellular telephone subscriptions are subscriptions to a public mobile telephone service that provide access to the PSTN using cellular technology.