The approach of I-CARD consists of the following:
Focus on broad regions of the country where economic vulnerability is particularly severe.
Distinguish rural and urban areas as patterns of vulnerability differ.
Identify and recruit block-level experts who can credibly and unbiasedly provide basic information concerning the humanitarian and livelihoods situation in specific blocks, with a focus on vulnerable groups, on an ongoing basis.
Construct a questionnaire covering multiple domains concerning humanitarian and livelihoods situation in a block: food & water, migrants (out-migrants, returned migrants, in-migrants), livelihoods, healthcare, access to cash and credit, and extent of social harmony or unrest. The questionnaire uses a simple 3-point scale for objectivity and comparability of information across blocks; however, every question also has a qualitative component where related remarks and details can be captured.
Assemble and train a team for conversations with each block expert to get information for the various parts of the questionnaire (on the 3-points scale and on qualitative remarks).
Collate and present block-level information on a monthly basis regarding the unfolding humanitarian and livelihoods situation in different blocks.
The validity of the data is based on the following factors:
Credibility of block experts and their understanding of block-level situation with focus on vulnerable groups
Closeness of block experts’ understanding of survey questions to I-CARD intentions
Ability of callers from I-CARD team to translate block experts’ answers to appropriate parts of the corresponding scale
Credibility of Block Experts: Block experts were nominated through recommendations from trusted individuals and CSOs based on pre-specified criteria, making nominees inherently credible. Further, an I-CARD team member had a full, separate discussion with each nominated block expert to gauge degree of experience and understanding of the block and any biases that may compromise proffered information; a structured, consistent protocol was followed for this. On the basis of this discussion, we did not recruit about 25% of nominees.
Understanding of Survey Questions: An I-CARD team member had a detailed and lengthy conversation with each recruited block expert, discussing economic and social layout of the block and specifics of vulnerable groups. Each question in the survey tool was discussed, along with the various dimensions to be considered in formulating each answer [Survey Questions}. Sufficient rapport is built with experts so that, where relevant, they are comfortable in saying that they are not confident about answering specific questions (based on further discussion, some of these could end up marked as “do not know”, thereby not compromising validity of the information).
Ability of Callers: Multiple discussions and training were held for I-CARD personnel to familiarize themselves with the nuances of the survey tool in a consistent manner. Regular debriefing sessions with I-CARD personnel making calls to experts are also planned; this is also expected to improve consistency and validity. The tool was itself circulated for comments from academicians and practitioners, and also piloted in several blocks. The final version of the tool incorporates learning from these comments and pilots, including the scaling used for the questions. Note that validity is increased in procuring information through conversations of block experts with trained team members rather than having block experts fill out the forms themselves. Further, the fact that they survey is longitudinal (information collection from the same expert on a monthly basis) increases validity over time as potential discrepancies can be discussed and redressed over time.
As a further validity check, I-CARD is striving to triangulate data by speaking with multiple block experts for the same block. At present (May 18th) we have information for 12 blocks where 2 block experts provided information separately and 1 block where 3 separate block experts were involved. The table below shows the summary of a validity check for the 12 blocks with 2 block experts each, for answers on a 3-point scale (averaged across various questions in the domains in the first column). Out of 12 blocks, overall only in 1 case were the block experts in full disagreement; in 7 blocks there were in full agreement. Further, even for specific domains (food, migrants, etc), the maximum disagreement between block experts only yielded 2 out of 12 blocks (for migrants and community questions).
The table below summarizes a validity check for the block with 3 block experts, focusing on the percentage of questions in each domain where each pair of block experts had maximum disagreement on the 3-point scale. For each pair, the overall disagreement was in the range 0-14% only. Note that the first and second block expert had no full disagreement except for one domain (community, 25%). The second and third experts had no full disagreements at all. And the first and third had full disagreements on three of the six domains, to the tune of 25-33%.
Maximum Disagreement among BEs on 3-point Scale
(% of Qs in each domain)