Jelke Bethlehem - Handbook of Web Surveys

Chapter 10is devoted to the problem of under‐coverage. This remains an important problem in many countries due to poor Internet coverage and the fact that Internet access is often unevenly distributed throughout the population. The chapter demonstrates how this can lead to survey estimates being biased. A number of techniques that may reduce under‐coverage bias are discussed.

Chapter 11examines self‐selection. The correct and scientifically well‐founded principle is to use probability sampling in order to select survey subjects and therefore allow reliable estimates regarding population characteristics to be calculated. Nowadays, it is easy to set up a web survey. Even those without any survey knowledge or experience can create one through dedicated websites. Many of the resulting web surveys do not apply probability sampling, but instead rely on self‐selection of respondents. This causes serious problems with estimation. Self‐selection and its consequences in terms of survey results are discussed in this chapter, demonstrating that correction techniques are not always effective, and there are many reasons why web‐survey‐based estimates are biased.

Nonresponse, under‐coverage, and self‐selection are typical examples, and adjustment weighting is often applied in surveys in order to reduce such biases. Chapter 12describes various weighting techniques, such as post‐stratification, generalized regression estimation and raking ratio estimation. The effectiveness of these techniques in reducing bias caused by under‐coverage or self‐selection is examined.

Chapter 13introduces the concept of response probabilities, describing how they can be estimated through response propensities. If estimated accurately, response probabilities can be used to correct biased estimates. Here, two general approaches are described: response propensity weighting and response propensity stratification. The first attempts to adjust the original selection probabilities, while the second is a form of post‐stratification.

Chapter 14is devoted to web panels. There are many such panels, particularly in the field of commercial market research. One crucial aspect is how the panel members (households, individuals, companies, and shops) are recruited. This can be carried out via a proper probability sample, or through self‐selection. There are consequences for the validity of the results of the specific surveys conducted with the panel members. The chapter discusses several quality indicators.

The accompanying website, www.web‐survey‐handbook.com, provides the survey data set for the general population survey (GPS), which has been used for many examples and applications in the book. The data set is available in SPSS (SPSS Corporation, Chicago, IL) format.

Silvia Biffignandi

Jelke Bethlehem

The editors acknowledge the contributions of:

Lon Hofman (Manager Blaise, Statistics Netherlands) and Mark Pierzchala (owner of MMP Survey Services, Rockville, USA) who wrote Section 1.3.1.

Annamaria Bianchi (University of Bergamo) and Barry Schouten (Statistics Netherlands) who wrote Chapter 8.

Web surveys are a next step in the evolution process of survey data collection. Collecting data for compiling statistical overviews is already very old, almost as old as mankind. All through history, rulers of countries used statistics to take informed decisions. However, new developments in society always have had their impact on the way the data were collected for these statistics.

For a long period, until the year 1895, statistical data collection was based on complete enumeration of populations. The censuses were mostly conducted to establish the size of the population, to determine tax obligations of the people, and to measure the military strength of the country. The idea of sampling had not emerged yet.

The year 1895 marks a fundamental change. Populations had grown bigger and bigger. It was the period of industrialization. Centralized governments required more and more information. The time was ripe for sample surveys. The first ideas emerged around 1895. There was a lot of discussion between 1895 and 1934 about how to select samples: by means of probability sampling or some other sample selection technique.

By 1934, it was clear that only surveys based on probability sampling could provide reliable and accurate estimates. Such methods of data collection were accepted as a scientific. In the period from 1940s to the 1970s, most sample surveys were probability based. Questionnaires were on paper forms. They were completed in face‐to‐face, telephone, or mail.

Somewhere in the 1970s another significant development started. The fast development of microcomputers made it possible to introduce computer‐assisted interviewing (CAI). This made survey data collection faster, cheaper, and easier and increased data quality. It was time in which acronyms like CATI (computer‐assisted telephone interviewing) and CAPI (computer‐assisted personal interviewing) emerged.

The next major development was the creation of the Internet around 1982. When more and more persons and companies got access to the Internet, it became possible to use this network for survey data collection. The first Internet surveys were e‐mail surveys. In 1989 the World Wide Web was developed. This software allowed for friendly graphical user interfaces for Internet users. The first browsers emerged and the use of Internet exploded. In the middle of 1990s, the World Wide Web became widely available, and e‐mail surveys were increasingly replaced by web surveys.

Web surveys are attractive because they have a number of advantages. They allow for simple, fast, and cheap access to large groups of potential respondents. Not surprisingly, the number of conducted web surveys has increased rapidly over time. There are, however, also potential methodological problems. There are ample examples of web surveys not based on probability sampling. Therefore, generalization of survey results to the population is questionable. The interviewed may access the Internet using various types of devices. Thus, web surveys can be completed and received not only on personal computer (PC) or laptop; it is highly probable the survey to be received in the mobile phone. The so‐called mobile web surveys are fully part of web surveys. This implies some methodological problems to be considered, and further research on the impact of mobile is called for.

This chapter describes the historical developments that have led to the emergence of web surveys. As an illustration, Section 1.3shows how these developments were implemented at Statistics Netherlands and led to new software for survey data collection.

The history of data collection for statistics goes back in time for thousands of years. As far back as Babylonian era, a census of agriculture was carried out. This already took place shortly after the invention of the art of writing. The same thing happened in China. This empire counted its people to determine the revenues and the military strength of its provinces. There are also accounts of statistical overviews compiled by Egyptian rulers long before Christ. Rome regularly took censuses of people and of property. The collected data were used to establish the political status of citizens and to assess their military and tax obligations to the state.

Censuses were rare in the Middle Ages. The most famous one was the census of England taken by the order of William the Conqueror, King of England. The compilation of his Domesday Book started in the year 1086 AD. The book records a wealth of information about each manor and each village in the country. Collected information was about more than 13,000 places. More than 10,000 facts were recorded for each country.