Improving web survey methodology for social and cultural research
Project leaders: Kea Tijdens and Paul Wouters
Postdoc researcher: Stephanie Steinmetz
Duration: 2008 - 2013
Links: www.wageindicator.org and the project description
Web surveys are currently being developed as a major tool of data collection in the social sciences, promising costs-effectiveness, quick data releases, large-scale data, and features such as highly flexible plug-ins. Recently, attempts have been undertaken to combine the advantages of both probability and volunteer web surveys, although the statistical theory behind mixed samples is not yet well developed. This three-year postdoctoral research project aims at methodological issues concerning web-surveys. In particular, it aims at furthering insight into bias in volunteer samples, the possibilities of propensity score weighting using a reference sample and the effects of other weighting procedures. In addition, it aims to improve insight into drop-out, item-non-response and optimal questionnaire length, using data of the large, continuous WageIndicator web-survey.
First year report
During the first year, empirical questions have been central to the project. As it was one of the aims to study the selection bias of the WageIndicator data and to develop propensity score weights for selected samples of the data set, the starting point was an in-depth literature review and the study of statistical techniques to develop such methods.
Based on these preparations, in a second step, two cooperations have been established with Spanish and US colleagues of the WageIndicator in order to develop a joint paper on the selection bias in Germany, the Netherlands, Spain, Argentina, the US and Brazil. This paper will be published as an AIAS working paper at the end of November 2009. Furthermore, a second paper has been developed on the selection bias and the development of propensity score weighting for Germany and the Netherlands using national reference surveys. Draft versions of this paper have been presented at international conferences (ESRA 2009, the 3rd MESS workshop) as well as in internal meetings (AIAS Lunch seminar). It will be published as an AIAS working paper (October 2009) and submitted to an international peer-reviewed journal (Survey Research Methods, November 2009).
As to the aim of the project to analyse the effect of web technologies on the community of sociologist and web-survey researchers, a broad literature overview provided the basis of the planned VKS book contribution (together with Paul Wouters and Clement Levallois) on ‘Sloppy data floods or precise social science methodologies? – dilemmas in the transition to data intensive research in sociology and economics’. This contribution will, besides other points, shed light on the discussion whether the technological changes in collecting and analysing data has led to a crisis in empirical sociology. In this context, earlier ideas have been presented at the Erasmus Studio seminar.
S.Steinmetz, D.H. de Vries, K.G. Tijdens, Should I Stay or should I go? The impact of working time and wages on retention in the health workforce (Amsterdam,2014).