Jelke Bethlehem - Handbook of Web Surveys

Under‐coverage occurs when the target population is wider than the number of people with Internet access. This leads to bias in estimates in the case of relevant differences between those with Internet access and those without.

Self‐selection is when a questionnaire is simply made available via Internet to all, with individuals nominating themselves.

A respondent is therefore anyone who happens to have Internet access, visits the website, and decides to take part in the survey. These participants generally differ significantly from nonparticipants.

General‐population surveys that aim to provide reliable and accurate statistics are traditionally carried out face‐to‐face or by telephone. Interviewers are used to persuade people to take part and to help respondents to provide the right answers. In web surveys, there is no interviewer assistance, a fact that can have serious impact on the quality of the data collected.

The diffusion of smartphones increases the possibility for interviewees to be reached via their mobile device and to have the questionnaire completed via the same device, resulting in the current trend in running mobile web surveys. Consequently, there are new risks for error in the survey due to device characteristics and the behavior of the user.

The researcher should have in mind that when a web survey is run a mobile web survey takes place, if questionnaire is not blocked against mobile devices. Here, for simplicity the term web surveys is used, meaninig the mobile web survey is included. Summing up, web surveys afford several challanges and need reseacher be conscious of the methodological issues for a good survey. At the time beeing, collecting data through web surveys is going to become a common practice both in market research, academic research and official statistics. Knowledge about how to manage a web survey, risks, errors and advantages is important.

This book provides an insight into the possible use of web surveys and mobile web surveys for data collection. Web surveys allow for lower data collection costs. It is also expected that web surveys lead to increased response rates. Is this the case? What about the quality of the data collected? This book examines many theoretical and practical aspects of mobile web surveys and can therefore be considered as a handbook for those involved in practical survey research, including survey researchers working with official statistics (e.g., in national statistical institutes), academics, and commercial market research.

The book's two authors have widespread expertise in survey methodology. They come from two different countries (the Netherlands and Italy) and different research organizations (a national statistical institute and a university). They therefore provide a broad view on the various theoretical and practical aspects of mobile web surveying.

The second edition of the book involves a revision of each chapter with the following criteria:

(1) to maintain the existing text and content as much as possible, (2) to update the existing text and content with comments based on new literature and results, and (3) to add new paragraphs (if necessary) to cover new relevant topics (see the contents and chapter description below). A number of new examples have been provided, some of the existing examples have been updated or substituted, and some applications have been replaced. Updates have also been included to highlight new trends in web surveys and emerging solutions. There are two new chapters on topics concerning mobile web surveys: one presenting a flowchart to illustrate the steps involved in running a survey via web and the other examining adaptive design. It was therefore necessary to renumber the chapters in respect to the first edition.

The first two chapters of the book provide an introduction into web surveys. Chapter 1provides a historic account of developments in survey research and shows how web surveys have become a tool for data collection. Section examines the Blaise system, which has been around for more than 30 years. New developments have taken place over the last 10 years, but no papers have been written on this subject. The section looks at the history and recent developments regarding Blaise; it was written by Lon Hofman and Mark Pierzchala and is published for the first time here.

Chapter 2is an overview of basic aspects of web surveys. It describes how and where they can be used. Official statistics departments, research institutions, market research companies, and private forums are all interested in web surveys studying both households/individuals and businesses.

Chapter 3presents a flowchart illustrating the steps (and sub‐steps) of web surveys, each accompanied by a short description. The flowchart is of potential use to both practitioners as a guideline for how the survey process should be carried out and to researchers in highlighting and explaining the positioning of their studies at the different steps of the survey process. It is also useful in discussing the errors that can occur in different steps. The chapter provides an introduction to the framework and its structure and discusses the relevance of bearing in mind the framework and the survey steps when considering web survey errors. It then goes on to describe the concept of the step and the structure of the flowchart, breaking down the web survey process into six main steps. These are analyzed in detail, and an overview of survey errors is provided.

Chapter 4examines the aspects of sampling. It is stressed that valid population inference is possible only if some form of probability sampling is used and that a proper sampling frame is required for this. A number of sampling designs and estimation procedures useful for web surveys are discussed.

A researcher conducting a survey may encounter a number of practical problems, and Chapter 5provides an overview of possible errors, with two types of error examined in further detail. The first concerns errors in measurement. These can be caused by specific issues in questionnaire design, as well as a number of other aspects such as technology, incorrect unit definition, and so on. The second type of error regards nonresponse. This is a phenomenon that can affect all surveys, but the specific aspects of nonresponse in web surveys require particular attention. The chapter provides advice on relationships between errors and information on the various types.

A web survey is just one form of data collection. There are others, such as face‐to‐face, telephone, mail, and mobile surveys. Chapter 6compares these various methods with online data collection, discussing the advantages and disadvantages of each one.

As web surveys do not involve interviewers, the respondents complete the questionnaire on their own. Furthermore, when questionnaires are sent out, they may very well be received and even completed on a mobile device (such as smartphones, which are very widespread). This means that questionnaire design is of crucial importance. Questionnaires must be adapted in order to be suitable for mobile devices; otherwise they cannot be used for this purpose. Small imperfections in the questionnaire may have serious consequences in terms of data quality. Questionnaire design issues are addressed in Chapter 7.

Chapter 8examines strategies for data collection with adaptive/responsive survey design. In this case, strategies are not defined in advance, but instead are adapted, if necessary, during fieldwork. These designs may contribute to countering growing problems of nonresponse. This chapter was written by Annamaria Bianchi and Barry Schouten, who applied their particular expertise in this field to the subject examined.

A web survey may not always be the best solution for providing reliable and accurate statistics, with quality being affected by problems of under‐coverage and low response rates. An interesting alternative is to set up a mixed‐mode survey, in which several data collection methods are combined either sequentially or concurrently. This approach is less expensive than a single‐mode interviewer‐assisted survey (conducted either face to face or by telephone) and solves under‐coverage problems, but at the same time it poses other difficulties, known as mode effects, with one of the most significant of these being measurement error. Mixing modes is also of critical importance, as is the fact that in practice, a web survey is always mobile, unless questionnaire access via mobile device is restricted. All these aspects, as well as others concerning mixed‐mode surveys, are discussed in Chapter 9.