Air raids, power cuts, and many, many questions ‐ collecting data at a time of war
Active conflicts make for a perilous backdrop for research, dangerous for researchers and survey respondents alike. Ongoing violence, landmines, checkpoints, and other restrictions prevent researchers from physically moving around to obtain a comprehensive picture of what is happening, making it difficult to determine what needs humanitarian response should address with priority, and where.
CATI as a Suitable Survey Modality
While various modalities are available for collecting data during ongoing wars, the specific situation, research questions asked, and available resources determine which modality is most appropriate for a given context. One approach proven particularly useful during ongoing conflicts is Computer-Assisted Telephone Interviewing (CATI), as it does not require researchers to be physically present, making research safer and feasible in situations where access is dangerous or limited. Furthermore, CATI provides respondents with a higher degree of anonymity, essential in contexts where individuals may fear retribution or be otherwise concerned with information-sharing.
Case Study: The General Population Survey in Ukraine
Since March 2022, the International Organization for Migration (IOM) and Multicultural Insights conducted the General Population Survey in Ukraine, completing 12 rounds of data collection throughout the country using a phone-based survey based on probabilistic sampling. The aim was to gather insights into internal displacement and mobility flows and assess needs. This blog article highlights some of the lessons that we learned from surveying over 24,000 respondents in Ukraine over the course of the first year of the full-scale war. The approaches developed can inform future research efforts in conflict settings, making information on the type and scale of needs among vulnerable populations known to those who may be able to assist.
- Interviewers Working from Home
To ensure continuous data collection even in regions with infrastructure issues or other challenges, it was crucial for Multicultural Insights, the firm contracted to collect data for the General Population Survey in Ukraine, to equip interviewers with the necessary tools to work from home. Having a pool of interviewers spread across various regions helped maintain data collection efforts, as those from areas with less disruption could continue making calls even if others were unable to work.
- Rotation of Interviewers and Managing the Emotional Toll
Dealing with emotionally charged responses can be challenging and potentially exhausting. Rotating interviewers periodically can help prevent burnout and ensure they can continue to carry out their tasks effectively. IOM and Multicultural Insights provided additional training to equip interviewers with the necessary skills to handle emotional situations during interviews, such as respondents crying or sharing sensitive information. IOM also provided psychological counseling to ensure interviewers were adequately supported and prepared to manage the emotional toll.
- Quality Control Measures and Feedback to Interviewers
IOM and Multicultural Insights refrained from recording interviews as obtaining permission to record would have led to decreased response rate. All quality control was based on live listen-ins. The quality control team constantly shared examples of difficult scenarios encountered during fieldwork and the best approaches used to address them. By sharing best practices and learnings with all interviewers in real time, we ensured that everyone was equipped with the knowledge and skills needed to conduct interviews successfully, resulting in high-quality data collection. Additionally,as the situation on the ground was constantly evolving, and new challenges were emerging, live listen-ins during einterviews provided valuable insights informing the content of future rounds of data collection.
- Question Sensitivity Redefined by War
Respondents' sensitivity towards certain topics shifted during the war. While questions concerning income and financial spending were previously considered extremely sensitive and therefore positioned at the end of the questionnaire, questions about the respondent's oblast of residence and demographic information about household members became even more sensitive during the war. To address these issues, we had developed additional explanations of the intended use of the data for targeting aid, as well as probing techniques to encourage respondents to share. Overall, sensitivity and compassion were prioritized in questionnaire design to ensure that the data collected is accurate, reliable, and respectful of respondents' emotional well-being. Enumerators also provided contacts to a free mental health hotline administered by IOM to over 5500 respondents who were interested in the service over the course of the year.
- Interviewers’ High Moral
The interviewers working on the General Population Survey continually demonstrated a high level of dedication and commitment to ensuring the accuracy and reliability of the data collected, driven by the humanitarian mission of the survey. During peacetime, survey quality control aims to detect fraudulent practices in survey responses. For instance, to speed up data collection, some interviewers may complete certain questions without consulting the respondents. Surveys conducted during times of war, however, tend to have higher levels of accuracy due to the strong ethical commitment of the interviewers. Our quality control thus focused on evaluating the interviewers' ability to comprehend questions, use effective probing techniques, and provide precise information to respondents who request or agree to receive assistance referrals. IOM and Multicultural Insights continually shared the survey-based research findings and publications with the interviewers to encourage their pride in the work done.
Conclusion
Data collection during an ongoing war is complex and challenging, yet also essential to ensuring that resources are allocated to those most in need. The General Population Survey served and continues to serve as a key data source for national level humanitarian coordination and planning in Ukraine in the aftermath of the full-scale invasion, including in the calculation of the number of persons in need and their profiles, and in the design of the Humanitarian Response Plan (HRP). Beyond adapting traditional CATI methods, the success of the survey ultimately demonstrated the commitment of the Ukrainian enumerators and respondents to persevere, collectively dedicating thousands of hours of time to asking and answering of detailed questions, while on their way to or sitting in basements, evacuating, or grieving their lost ones, in hope of contributing in a small way to collective resilience.