Jelke Bethlehem - Handbook of Web Surveys

A second type of configuration is when there exists overlap between the different data sources. The overlap can concern the units, the measured variables, or both.

A third situation is when the combined data entail under‐coverage of the target population in addition, even when the data are in an ideal error‐free state.

A further configuration is when microdata and aggregated data are available. There is overlap between the sources, but there is the need to reconcile the statistics at some aggregated level. The conciliation can be achieved by means of calibration, which is a standard approach in survey sampling. Of particular interest is when the aggregated data are estimates themselves.

Finally, it is possible that multisource approach refers to longitudinal data. More questions arise; the most important issue is that of reconciling time series of different frequencies and qualities. For example, one source has monthly data and the other source has quarterly data.

Integration may occur between different types of sources: surveys (mainly web surveys), administrative data, other passive collected data, social network, and other unstructured data.

Integration of different configurations as well as of different types of data sources implies different methodological problems. For instance, integrating survey and administrative data through unit record linkage requires improving coherence across data collections, using standard classifications and questions, rationalizing content between surveys, and processes for combining separate sample surveys into one survey vehicle (Bycroft, 2010).

Example 1.7discusses an integration of web scraped information, administrative data, and surveys, whereas Example 1.8shows an application of integration between survey data and social network unstructured information.

As a result of the multisource integration, statistical output is based on complex combinations of sources. Its quality depends on the quality of the primary sources and the ways they are combined. Some studies are investigating the appropriateness of the current set of quality measures for multiple source statistics; they explain the need for improvement and outline directions for further work.

Istat since 2015 has been experimenting web scraping, text mining, and machine learning techniques in order to obtain a subset of the estimates currently produced by the sampling survey on “Survey on ICT Usage and e‐Commerce in Enterprises” yearly carried out on the web. Studies from Barcaroli et al., (2015, 2016), and Righi, Barcaroli, and Golini (2017) have focused in implementing the experiment and in evaluating data quality.

Trying to make the optimal use of all available information from the administrative sources to web scraping information, web survey estimates produced could tentatively be improved. The aim of the experiment is also to evaluate the possibility to use the sample of surveyed data as a training set in order to fit models to be applied to website information.

Recent and in progress steps are a further improvement to the performance of the models by adding explicative variables consisting not only of single terms, but the joint consideration of sequences of terms relevant for each characteristic object of interest. When a certain degree of quality of the resulting predictive models will be guaranteed, their application to the whole population of enterprises owning a website will be performed. A crucial task will be also the retrieval of the URLs related to the websites for the whole population of enterprises. Finally, once having predicted the values of the target variables for all reachable units in the population, the quality of estimates obtained will be analyzed and compared with the current sampling estimates obtained by the survey. In a simulation study, Righi, Barcaroli, and Golini (2017) found that the use of auxiliary variable coming from the Internet DB source highly correlated with the target variable does not guarantee enhancement of the quality of the estimates if selectivity affects the source. Bias may occur due to absence of some subgroups. Thus an analysis of the DB variable and the study of the relationship between populations covered or not by the DB source is a fundamental step to know how to use and which framework implement to assure high‐quality output.

In conclusion, the approach that uses web scraping and administrative data together with the web survey looks to be promising; nevertheless quality results of the estimations are satisfactory only in some cases. The use of big data has to be carefully evaluated, especially if selectivity affects the source.

Example 1.8focuses on an experiment of integration of social media data and surveys. Even if the study lacks statistical representativeness and indicators, it presents an interesting approach that should be deeply investigated and statistically formalized.

Wells and Thorson (2015) introduce a novel method that combines a “big data” measurement of the content of individuals' Facebook (FB) news feeds with traditional survey measures to explore the antecedents and effects of exposure to news and politics content on the site. This hybrid approach is used to untangle distinct channels of public affairs content within respondents' FB news feeds.

The authors explore why respondents vary in the extent to which they encounter public affairs content on the website. Moreover, they examine whether the amount and type of public affairs content flows in one's FB are associated with political knowledge and participation above and beyond self‐report measures of news media use.

To combine a survey with measurements of respondents' actual FB experiences, they created a FB application (“app”) and embedded it within an online survey experience.

Respondents, undergraduates at a large Midwestern public university, visited a web page and gave two sets of permissions: they first consented to be participants in a research study—a form required by the institutional review board—and then they separately approved the app through their FB profile. Once they approved the app, they were returned to the survey to complete the questionnaire. While respondents completed the questionnaire, the app recorded specific elements of their FB experience (with respondents' permission), such as how many friends they had, what pages they followed, and what content appeared in their news feeds during the previous week. When respondents had completed the survey, the app had finished its work and automatically removed itself from respondents' profiles. This research was approved by a standard university institutional research board and was designed to comply with FB's Platform Policies and Statement of Rights and Responsibilities, each of which placed restrictions on the use and presentation of the data.

The resulting database offers an original combination of respondent's self‐reported attitudes and media behaviors (including FB experience) with measure of part of their FB experience.

From the statistical point of view, the study has limitations (Beręsewicz et al., 2018; Biffignandi and Signorelli, 2016). The empirical study is run on a small sample of college volunteers. Thus, they have no claim of representativeness. In addition they have considered only a single information platform (FB). Other limitations suggest to consider the results just as a first experimental research. However, the approach proposed is in line with interesting methodological innovations toward the combination of social media trace with conventional methods. It opens the perspective to better understand big data and then try to relate big data descriptive information to socioeconomic theoretical hypotheses.