Ethiopia

 The simplest method of collecting EquityTool data is to sign up to our web app. To use the EquityTool in DHIS2 or another data collection platform, you will need to download the supporting file. Click on your preferred data collection method and complete the form to receive the file via email. Please check your junkmail folder if you do not receive an email from us.

DHIS2

Agree to the Terms and Conditions


Other platforms

Agree to the Terms and Conditions


 

EquityTool: Update released Jan 5, 2018

The EquityTool has been updated based upon new source data. The original version is no longer active but is available upon request.

Previous version Released December 9 2015

                       

Source data: Ethiopia DHS 2016

 

# of survey questions in full wealth index: 37

# of variables in full index: 108

 

# of survey questions in EquityTool: 15

# of variables in EquityTool: 17

 

Questions:

Question Option 1 Option 2 Option 3
DETERMINE IF THE RESPONDENT LIVES IN AN URBAN OR RURAL AREA

 

 

Urban Rural
Q1 Does your household have… electricity? Yes No
Q2 … a radio? Yes No
Q3 … a television? Yes No
Q4 … a refrigerator? Yes No
Q5 … an electric mitad? Yes No
Q6 … a table? Yes No
Q7 … a chair? Yes No
Q8 … a bed with cotton/sponge/spring mattress? Yes No
Q9 Does any member of this household have a bank account? Yes No
Q10 What is the main source of drinking water for members of your household? Piped to yard / plot Other
Q11 What kind of toilet facility do members of your household usually use? Pit latrine without slab / open pit No facility / bush / field Other
Q12 What type of fuel does your household mainly use for cooking? Electricity Wood Other
Q13 What is the main material of the floor in your household? Earth / sand Other
Q14 What is the main material of the exterior walls in your household? Bamboo with mud Other
Q15 What is the main material of the roof in your household? Metal / corrugated iron Other

 

Technical notes:

Recreating the full index

To create the EquityTool, we simplify the original, full wealth index that is found in the relevant DHS dataset. In the case of Ethiopia DHS 2016, we did not have access to the factor weights from the full wealth index. This led us to recreate the full wealth index, using a process in line with guidance from ICF. We used this recreated wealth index as the basis for the scoring we used in our EquityTool analysis.

Separate urban and rural indices to create a national EquityTool

We were unable to achieve agreement of kappa>=0.75 between the full DHS wealth index and a simplified index using our standard simplification process (detailed in this article). Using a revised approach, detailed below, high agreement (kappa>=0.75 for both urban and national indices) was achieved. The data used to identify important variables came from our recreated full wealth index described above.

We were unable to achieve a reduction in questions or an agreement of kappa>=0.75 between the original DHS wealth index quintiles and quintiles created using factor weights from the national factor weights for Ethiopia DHS 2016. These national factor weights come from an analysis of the national population, and contain only those variables which are related to the construct of wealth in the same way in both rural and urban areas.  These national factor weights are usually used in EquityTools to calculate national quintiles, as they reduce some known areas of respondent error in the survey.

To overcome this problem of low agreement, we instead used the factor weights from the rural and urban analyses, which select variables that relate to wealth differently in urban and rural areas. For example, in an urban area, ownership of chickens may be associated with being relatively poor, while in rural areas, it may be associated with being relatively wealthy. This is the case in Ethiopia. A short list of variables, common to both urban and rural areas, are iteratively selected to find those which result in high agreement (kappa>=0.75) against the original wealth index quintiles for national and urban populations. For Ethiopia, the scores for urban and rural residents were combined into a national score using linear regression, in a process similar to that used by ICF. Specifically, a score from the simplified index for urban residents (Uscore) was regressed against the wealth index score variable created for the recreated full wealth index analysis (Nscore), the same was done for rural residents (Rscore), and the resulting coefficients are used to create a single national score (NatScore).

 

Nscore=b1Uscore + a1

Nscore=b2Rscore + a2

NatScore=b1(Uscore)(Urban)+ a1(Urban)+b2(Rscore)(Rural)+a2(Rural)

Where Urban=1 if respondent lives in urban area and 0 if otherwise, and Rural =1 if respondent lives in rural area and 0 if otherwise.

Respondents' quintile assignments resulting from NatScore, the national wealth index score created from a simplified list of questions, were compared to the quintile assignments resulting from the original wealth index with 108 variables using the kappa statistic.

The questions in the simplified index which resulted from this process differ from our standard approach. We need to know whether the respondent lives in an urban or rural area. An additional question was added to the EquityTool: 'DETERMINE IF THE RESPONDENT LIVES IN AN URBAN OR RURAL AREA'. In principle, the definition of 'urban' and 'rural' should match the definition used in the Ethiopia DHS 2016. In reality, the user needs to decide how to determine if each respondent lives in an urban or rural area. Three approaches are presented below, with some notes on each. Whichever method is chosen, it should be uniformly applied across all surveys conducted.

  1. Ask the respondent directly – 'is your home in an urban or rural area'. This relies on the respondent's understanding of what 'urban' and 'rural' is.
  2. Allow the data collector to determine, based on guidance provided. This will work best if interviews take place in or very near to people's homes, and if the data collectors can be trained on the same rules to determine if an area is urban or rural. One example of a rule is to classify 'peri-urban' areas on the edges of a city or town as urban. Another rule might be to classify an area as urban if it has a market center which operates daily.
  3. If the interviews are taking place outside the home, then classify respondents based upon the location of the interview. For example, if interviews occur in health facilities, classify respondents as urban if the facilities are located in urban areas. Individuals may travel, so this method is also subject to error.

 

Level of agreement:

National Population

(n=16,650)

Urban only population

(n=5,232)

% agreement 84.2% 84.5%
Kappa statistic 0.755 0.757

 

Respondents in the original dataset were divided into three groups for analysis – those in the 1st and 2nd quintiles (poorest 40%), those in the 3rd quintile, and those in the 4th and 5th quintiles (richest 40%). After calculating their wealth using the simplified index, they were again divided into the same three groups for analysis against the original data in the full DHS. Agreement between the original data and our simplified index is presented above.

What does this mean?

When shortening and simplifying the index to make it easier for programs to assess equity, it no longer matches the original index with 100% accuracy. At an aggregate level, this error is minimal, and this methodology was deemed acceptable for programmatic use by an expert panel. However, for any given individual, especially those already at a boundary between two quintiles, the quintile the EquityTool assigns them to may differ to their quintile according to the original DHS wealth index.

The graph below illustrates the difference between the EquityTool generated index and the full DHS wealth index. Among all of those people (20% of the population) originally identified as being in the poorest quintile, approximately 83.5% are still identified as being in the poorest quintile when we use the simplified index.  However, approximately 12.5% of people are now classified as being in Quintile 2.  From a practical standpoint, all of these people are relatively poor. Yet, it is worthwhile to understand that the simplified index of 15 questions produces results that are not identical to using all 37 questions in the original survey.

The following table provides the same information on the movement between national quintiles when using the EquityTool versus the original DHS wealth index:

    EquityTool National Quintiles
  Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 Total
Original DHS National Quintiles Quintile 1 16.70% 2.50% 0.40% 0.20% 0.20% 20%
Quintile 2 4.10% 12.30% 3.30% 0.20% 0.10% 20%
Quintile 3 0.10% 4.10% 12.50% 3.10% 0.10% 20%
Quintile 4 0.00% 0.10% 3.70% 14.60% 1.60% 20%
Quintile 5 0.00% 0.00% 0.00% 1.90% 18.00% 20%
Total 20.90% 19.10% 19.90% 20.00% 20.10% 100%

 

The following graph provides information on the movement between urban quintiles when using the EquityTool versus the original DHS wealth index:

The following table provides the same information on the movement between urban quintiles when using the EquityTool versus the original DHS wealth index:

 

    EquityTool Urban Quintiles
  Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 Total
Original DHS Urban Quintiles Quintile 1 14.90% 4.30% 0.80% 0.00% 0.00% 20%
Quintile 2 5.10% 11.50% 3.20% 0.20% 0.00% 20%
Quintile 3 0.00% 4.10% 12.20% 3.60% 0.10% 20%
Quintile 4 0.00% 0.00% 3.50% 13.80% 2.70% 20%
Quintile 5 0.00% 0.00% 0.00% 2.80% 17.20% 20%
Total 20.00% 20.00% 19.60% 20.50% 20.00% 100%

 

Data interpretation considerations:

  1. This tool provides information on relative wealth – 'ranking' respondents within the national or urban population. The most recent available data from the WorldBank indicates that 33.5% of people in Ethiopia live below $1.90/day[1]. This information can be used to put relative wealth into context.
  2. People who live in urban areas are more likely to be wealthy. In Ethiopia, 89.2% of people living in urban areas are in the richest national quintile, compared to only 7.3% of those living in rural areas[2].
    1. If your population of interest is predominantly urban, we recommend you look at the urban results to understand how relatively wealthy or poor they are, in comparison to other urban dwellers.
    2. If the people you interviewed using the EquityTool live in rural areas, or a mix of urban and rural areas, we recommend using the national results to understand how relatively wealthy or poor they are, in comparison to the whole country.
  3. Some states in Ethiopia are wealthier than others. It is important to understand the country context when interpreting your results.
  4. In most cases, your population of interest is not expected to be equally distributed across the five wealth quintiles. For example, if your survey interviewed people exiting a shopping mall, you would probably expect most of them to be relatively wealthy.

Changes from the previous EquityTool

We released an EquityTool on December 9th 2015 which compared user data to a benchmark of DHS 2011.  A new source survey, the DHS 2016 was recently released, and allows us to benchmark results to a more recent population.  This is important, because wealth generally increases over time, and comparing your respondents to an old benchmark population will lead to over-estimating the relatively wealthy in your survey. In generating the new EquityTool, no attempt was made to account for the fact that a previous version existed. In other words, we did not explicitly try to keep the same questions or response options as the previous tool.

For those who have not previously conducted an EquityTool based study in Ethiopia, the remainder of this section is not particularly relevant.  For those who have used the previous EquityTool, you may be interested to know how the two versions compare.

 

Previous Current
Source Data DHS 2011 DHS 2016
# of questions in EquityTool 14 15
# of questions in full wealth index 36 37
Kappa statistic (EquityTool vs full wealth Index) for 3 groups National: 0.755

Urban: 0.757

National: 0.76

Urban: 0.771

 

Practical considerations for users of the previous EquityTool

Comparing the results of surveys that used the previous EquityTool against those that use the current EquityTool is difficult. It will not always be clear whether any difference is because of actual differences in the wealth level of the respondents or because the EquityTool has changed.

The technical comparison section below, particularly the 3rd comparison, illustrates how quintile results compare when using the previous EquityTool and the current one. Generally, there is a partial shift down in quintiles when using a more recent EquityTool. In other words, the current EquityTool will usually put some respondents into a lower quintile than the previous one would.

It is generally best to use the current version of the EquityTool, since it will give a more accurate quintile estimates. If you are currently collecting data with the previous tool, it is best to continue to use the previous tool. Note that if you have created a survey in the EquityTool web application using the previous EquityTool, that survey will continue to use the previous EquityTool.

If conducting a follow-up survey to a baseline that used the previous EquityTool, and the most important result is change from the baseline, it may be preferable to continue to use the previous EquityTool for comparability. If you need to do this, please contact us at equitytool@m4mgmt.org.

 

Technical comparison between the current and previous EquityTool

All of the questions and response options for the previous EquityTool are found in the new source data (DHS 2016). This makes comparison between the two versions of the EquityTool, and two different data sources, easier.

The comparison will be assessed in 3 different ways, described below.

 

  1. Using the same 14 questions and response options, and scoring system as in the previous EquityTool, with two different benchmark populations.

This analysis simulates results if the only thing which changes is the benchmark against which respondents are compared. In the 5 years between the two source data studies, more people have acquired assets that are indicative of wealth. In the graph below, the previous EquityTool, derived from the 2011 DHS, is applied to the 2011 DHS data and the newer 2016 DHS data. In 2011, the proportion of households in each of the 5 quintiles is very close to 20%. However, by 2016, the distribution is skewed towards the wealthy. People become wealthier overall.

We do not use the previous questions and weights, because over time, the population has become wealthier. Thus, comparing your respondents to this skewed distribution becomes challenging.

  1. Keeping the same 14 questions and response options as the previous EquityTool, but calculating scores based upon the 2016 data.

As an alternative, one might wish to use the same questions as the previous tool, but update the weighting. This seems reasonable, as the relative contribution of each asset towards overall wealth may have changed over time. Using new weights, but the same variables as the previous tool, we can see how well the resulting quintiles compare to the quintiles based on the full wealth index created by ICF.

 

The table below presents the agreement between the quintiles created from the full wealth index in the DHS 2016 dataset and the quintiles created by the previous EquityTool, the previous EquityTool variables with updated weighting, and the current EquityTool. As with the agreement statistics above, these figures are for the bottom 2 quintiles, middle quintile and top 2 quintiles.

 

2011 EquityTool 2011  questions, 2016  scoring 2016 EquityTool
Agreement 69.5% 79.4% 84.2%
Kappa 0.518 0.678 0.755

 

The current EquityTool has the best agreement with the full wealth index quintiles and is the only one that exceeds our minimum kappa statistic of 0.75. The previous tool, even when the scoring is updated, falls short of this standard. The reason for this difference is because these 14 questions from the previous tool are no longer the best predictors of the overall wealth distribution.

  1. Comparing the previous 14 questions and scores, and the new EquityTool 15 questions

The table below shows how the previous and current EquityTool compare, using the same population. This is analogous to a comparison of the two versions of the EquityTool on the population you surveyed using our previous EquityTool.

    Previous EquityTool Quintiles
  Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 Total
Current EquityTool Quintiles Quintile 1 9.4% 8.7% 2.3% 0.5% 0.0% 20.9%
Quintile 2 1.7% 4.9% 7.2% 5.2% 0.1% 19.1%
Quintile 3 0.0% 0.9% 6.3% 12.0% 0.7% 19.9%
Quintile 4 0.0% 0.0% 0.4% 9.7% 9.9% 20.0%
Quintile 5 0.0% 0.0% 0.0% 0.5% 19.5% 20.1%
Total 11.1% 14.5% 16.2% 28.0% 30.2% 100.0%

 

The rightmost column indicates that the current EquityTool does in fact evenly divide the population into 5 groups.  The bottom row shows that using the older EquityTool does not divide the population into equal quintiles – it puts more people into the higher quintiles. The cells within the table indicate how respondents are categorized, if measured using the two different tools. Of those who are categorized as quintile 1 (poorest) using the current tool, only 45% of them would have been considered in quintile 1 in the previous tool (see the first row). Similarly, for those currently categorized as in quintile 3, 60% would have previously been categorized as being in quintile 4. If you had used the previous EquityTool, you can expect that with the current version, your respondents will look more poor.  This is not incorrect, but rather reflects the reality that we are measuring them against a more accurate benchmark.

Metrics for Management provides technical assistance services to those using the EquityTool, or wanting to collect data on the wealth of their program beneficiaries. Please contact equitytool@m4mgmt.org and we will assist you.

 

[1] From povertydata.worldbank.org, reporting Poverty headcount ratio at $1.90/day at 2011 international prices.

[2] From the Ethiopia DHS 2016 dataset household recode, available at http://dhsprogram.com/