[Frannie Noble]

Our Save the Children Research Specialist, Jonathan Seiden, replied:

I would say it depends on your research questions. What are you trying to answer by collecting this data? Having said that, 120 is a pretty small sample, so I would probably recommend doing a full census of them.

[Frannie Noble]

No, IDELA is an open source tool available to the public free of cost. Save the Children does ask IDELA users to sign a Memorandum of Understanding (MoU) to clarify how IDELA should be used and to encourage partners to share information and learning about ECCD. Information about the MoU is available on the IDELA website.

[Frannie Noble]

And the person who originally posted the question responded,

Yes, thanks a lot for the sampling discussion. Yes, we too were thinking of doing a census. The purpose of the assessment is kind-of measurement of the effectiveness of the ECCD classes and school readiness of those children to enter the school in September 2018.
Considering the difficulty of gathering all parents in the villages (due to seasonal works in the fields, taking care for other children at home, etc.) we will reach out as much parents as possible to ensure at least the highest respondent rate closest to the possible rate the census could provide.

[Frannie Noble]

Lauren Pisani, a Research Advisor with Save the Children, also replied:

Ideally we want the children and parents to be a 1-to-1 match. So for each child, you assess one caregiver. This allows you to look at many different aspects of equity in your analysis.

[Frannie Noble]

This response comes from Lauren Pisani, an Advisor for Learning Research at Save the Children and an IDELA expert.

I suppose it could but this would be difficult to observe from a group of children. You would need highly trained observers and the process would benefit from individual interviews with each child. I don’t think you would get all that you want to know about children’s emotional wellbeing from a group observation.

[Jonathan Seiden]

Hi Sarah,

I’d recommend using KoBo over Tangerine for administering IDELA. The XLSForm that is available here is easily adaptable is already optimized for use in KoBo. We may be updating those templates to a more user-friendly version, but they work perfectly fine as-is. XLS forms make adding additional languages very easy as you just have to translate the “labels” for questions and choice answers.

It’s quite common to have the instructions in a colonial language and then the child-facing language in a local language. Just ensure you translate the words in **bold** into local language.

If you make a French version in XLSForm, we’d love for you to share the final version so that others can use it in the future.

[Fabiola Lara]

Translation is important as children need to understand what they are being asked but adaptation for contextual relevance is equally important. It is highly advisable that an expert group reviews the IDELA tool well in advance of the first IDELA training. No major adaptations should take place during this time as this affects reliability. However, things like adapting the greeting and consent section, changing animals/fruits mentioned in the tool to others more familiar to children based on their surroundings, etc. is fine.

[Fabiola Lara]

Yes, it is important that all enumerators are delivering the items to children in the same way to achieve standardization and ultimately, reliable data. Enumerators should memorize items for smooth delivery of assessment when assessing with young children.

[Fabiola Lara]

The core IDELA tool does not include indicators on learning involvement or child-wellbeing, but we do try to address these important topics in a few different ways. The IDELA tool has optional items about children’s learning approaches (persistence, curiosity, attention, etc.). The items have not been validated as part of the core tool, but we are continuing to test and analyze their utility. In addition, we encourage teams to complement the IDELA tool with other measures that do measure these topics. For example, IDELA has been alongside classroom environment tools like the ECERS as well as child well-being measures that address children’s stress and psychosocial wellbeing.

[Fabiola Lara]

Response from Jonathan Seiden, Learning Research Specialist:

Currently, the methodology for creating domain-level and overall scores is quite simple.

1) Calculate the %correct for each activity/task. Take the number of correct responses and dividing by the total possible. Note that some tasks are 0 and 1, whereas other tasks have different number of points possible. For example, for the Drawing a Triangle task, there are two items. For the first item (how many closed corners) a child can get up to 3 with possible scores 0, 1, 2, and 3, and for the second item (does it look like a triangle), the scores are 0 for incorrect and 1 for correct. So the total possible points for this task are 4.

2) Calculate the Average % correct for each domain. Simply sum the % correct for each of the TASKS in the domain, and then divide by the number of tasks. So for the motor domain, you take (fold%correct + drawtriangle%correct + drawperson%correct + hopping%correct) / 4

3) For the overall IDELA, simply take the average of all of the 4 core domain scores (Emergent Literacy, Emergent Numeracy, Motor, and Social-Emotional).

Please note that there is a .do file if you plan to analyze data in Stata, and also an Excel template which will automatically calculate these scores for you. You can find them in the “Training, Data Collection and Analysis Resources” section of the “The IDELA Tool” page: https://idela-network.org/the-idela-tool/training-data-collection-analysis-resources/

[Sarah Strader]

Hi Jonathan,

Thanks so much for this tip! How easy is KoBo to program for someone that has never done software programming before?

Thanks,

Sarah

[Jonathan Seiden]

Hi Sarah,

There are a lot of good tutorials on KoBo online. The official channel is here (with lots of great quick tips): https://www.youtube.com/channel/UC2v5y_Ln1IeH1W49lOj3vYg

There are some longer videos online as well. But with IDELA, it’s pretty easy, because the XLSForm is already built, and you just have to add the translations into each question!

[Frannie Noble]

Hi Sarah, We don’t have specific tablets that we endorse, but hope that the guidance below helps a bit: It comes from our Learning Research Specialist, Jonathan:

KoBo has this available; http://help.kobotoolbox.org/collecting-data/recommended-devices-for-data-collection

I’d recommend that tablets have at least 2GB of RAM and a 1.3GHz processor.

[Fabiola Lara]

With an impact evaluation, you are typically trying to determine whether an intervention has had a significant effect on the treatment group compared to some kind of control or comparison group. You need a sample size that is large enough to conduct the statistical tests to determine the magnitude of the difference between the two groups.

[Fabiola Lara]

Probing and repeating are important when administering the IDELA, but it is equally important to understand when and how much is appropriate. Too little probing/repeating might bias results towards an incorrect response. A child could have known the answer but not understood the question or responded to the wrong question. Too much probing/repeating might bias results towards a correct response. A child who receives many opportunities to respond may be helped by the extra chances. In short, probing should be used to clarify your understanding of a child’s response.

[Author]

Replies