Abstract
Background: There was an overwhelming demand for data to respond to economic and health emergencies during the COVID-19 pandemic. This forced the remote modes of data collection such as mobile and web surveys to come to the forefront, which was not the case before in many low and middle-income countries, including India. The primary concerns with remote mode surveys are undercoverage of target population and self-selection of the survey respondents resulting in biased estimates.
Methods: Using unit level data from COVID-19 Trends and Impact Survey (CTIS) from India, the largest public health web survey, we examine the bias in the estimates of vaccine uptake, a population measure which changes rapidly with time, particularly right after its roll out in India on 16 January 2021. In the absence of independently verified ‘ground truth’ or ‘gold standard’ for assessing bias in surveys, we discuss the need for statistical representativeness of web surveys and methods of achieving it.
Results: Bias in CTIS estimates of vaccine uptake is not constant over time, rather it increases up to a certain point of time and then decreases. Our findings are explained by the fact that the variability in the outcome of interest in the population first increases with time and then goes downward after more than 50% of the population are vaccinated. The validation of CTIS vaccine uptake estimates was possible as it is one of the rare situations where reliable gold standard measures were available. For another key indicator from CTIS, COVID-like illness (CLI) constructed based on self-reporting of symptoms, it is not trivial to assess the bias in the outcome as the quality of the gold standard is questionable.
Conclusion: Since absence of independently verified ‘ground truth’ or ‘gold standard’ for assessing bias in surveys is well acknowledged, it is crucial to validate statistical representativeness of web surveys with respect to key demographic characteristics of respondents which are often correlated with many outcome variables.