IDELA Endline Report
This briefer summarizes a quasi-experimental impact evaluation of Save the Children’s (SC) two-year sponsorship-funded preschool program in Meherpur, Bangladesh.
This briefer summarizes a quasi-experimental impact evaluation of Save the Children’s (SC) two-year sponsorship-funded preschool program in Meherpur, Bangladesh. The program combines two years of preschool with monthly parenting and Reading for Children sessions. Meetings between teachers and parents, and workshops on developing low-cost learning materials for children also feature in the program.
In February-March 2015 a baseline was administered to 258 intervention children and 240 comparison children using the International Development and Early Learning Assessment (IDELA), alongside a questionnaire for the primary caregivers of all 498 children. A follow-up assessment and caregiver questionnaire was administered to as many of the same children and caregivers as could be found in December 2015, after about one school year had passed.
At endline, with the possible exception of approaches to learning, there is still much room for improvement in children’s development in all domains. However, children who participated in the preschool program increased their development scores significantly more than comparison children in all four core IDELA domains, with an overall IDELA effect size of 0.68. Table 1 shows the effect sizes by domain, and Figure 1 represents these results graphically.
Table 1: Effect Sizes by Domain
|Motor Skill Development||0.52|
|Executive Function||Not significant|
|Approaches to Learning||Not significant|
The emergent literacy domain was children’s weakest domain at baseline and showed the least progress of all the core domains. The two additional domains of executive function and approaches to learning did not show a statistically significant difference in growth between intervention and comparison children.
However, at baseline children in the intervention group were significantly advantaged in their early learning environments and development relative to children in the comparison group. These baseline differences, and the uncertainties arising from the preschool enrollment and baseline sampling strategy, make it difficult to attribute endline learning gains to the two-year preschool program.
Nonetheless, Figure 2 presents evidence that the program increased learning material availability. This increase in materials was positively correlated with children’s learning gains. Given this, as well as the sheer magnitude of the additional learning gains of the intervention group over the comparison group even after controlling for the advantages of the intervention group, it is likely that at least part of these gains come from the impact of the preschool program.
A more rigorous research design is necessary to verify this. Future pilot studies should carefully structure the approach to enrollment and sampling to maximize confidence in impact evaluation results, as well as collect additional information on beneficiary participation and program quality in order to correlate this information against children’s IDELA gain scores.
In terms of equity, at baseline the following types of children scored lower on IDELA:
- children from larger families
- children from families with less reading materials and toys
- children who experienced more negative discipline
- children whose primary caregiver expressed less positive attitudes about their role in their child’s development
- children from poorer households
The only equity issue potentially addressed by the preschool program was the disparity between children from print-rich households versus print-poor households in emergent literacy development. Otherwise, among both intervention and comparison groups inequities tended to grow over time, especially in socio-economic status. Boys and girls gained equally on all domains at baseline and endline. These are important challenges for Save the Children to address in order to reach the most disadvantaged and most marginalized children.
- Considering programmatic implications, this analysis suggests several important areas of focus:
- In future program impact evaluations, carefully structure the approach to enrollment and sampling to maximize confidence in impact evaluation results and collect additional information about variation in participant participation and program quality
- Diversify the types of print children have access to, especially coloring books and comics.
- Expose children to the types of toys they do not have access to at home (especially puzzles, coloring oys, counting toys, etc.)
- Provide opportunities for caregivers to practice strategies for improving the frequency and quality of learning and play interactions in the household
- Include messaging about the harmful effects of this discipline as well as suggest positive discipline strategies for parents to try out.
- While addressing all domains and skills, focus more on improving children’s fine motor skills, letter knowledge, expressive vocabulary, oral comprehension, numbers and counting, spatial reasoning and pattern recognition behind sorting and puzzle tasks, length discrimination, linking shapes to the environment, and simple operations, and emotional self-awareness
- Brainstorm strategies for ensuring that the poorest children have access to quality learning in formal settings as well as in the household. Support all parents from these communities, especially those who are the most disadvantaged
- Work with families to create additional toys and reading materials for children