Development

Deciding what to measure, and how to measure it

The EquityTool you see today is the result of several years of collaborative work by researchers, donors and program implementers, searching for a comparable way to measure the socio-economic status of clients of health clinics.  The search was initiated by PSI and MSI, who wished to standardize the way they measured the wealth of the client populations they were serving in different countries.

After considering three alternatives, and pilot testing two of them in six different countries, the Social Franchising Metrics Working Group, hosted at the University of California San Francisco, adopted the use of the full Demographic and Health Survey (DHS) wealth index to measure wealth and developed a toolkit to help NGO programs use it. 

Demographic and Health Surveys
DHS are large scale national surveys, supported by USAID. The surveys include questions about the respondent’s household – the assets owned and the characteristics of the household. These data are used to generate a measure of socio-economic status known as the wealth index. Each respondent is assigned a wealth index score that indicates their wealth relative to the rest of the population. Using this score, the population can be sorted into categories of relatively poor through relatively rich. There are five categories, each containing 20% of the population. Thus, they are referred to as ‘wealth quintiles’. The late 1990s and early 2000s saw an increasing awareness of the relationship between health and wealth, as well as the ability to systematically analyze large national surveys to understand the degree of inequity in health and education indicators between rich and poor.  

Detailed information about the DHS wealth index can be found on the DHS website here. Learn more about wealth quintiles and relative wealth here.

By adopting the DHS Wealth Index as a standard measure of wealth, programs could theoretically survey their beneficiaries, and make evidence-based decisions about how to reach those in need.  However, the tool proved too unwieldy to be easily used in the field, and too complex for a non-specialist to analyze. The need for a simplified, easy-to-use and accurate tool for wealth assessment was evident.

Thoughtful simplification

In 2015, Population Services International (PSI) gathered a panel of experts in the field of wealth measurement and the DHS wealth index together. The panel included representatives from USAID, PSI, Marie Stopes International, Results for Development, BroadBranch, and Metrics for Management (M4M) and was tasked with reviewing alternate approaches to wealth measurement.

The panel discussed several alternative simplification approaches, and ultimately agreed on one index that now serves as the basis for today’s EquityTool. The chosen approach has three priorities:

  1. Ensure a minimum burden on the survey respondent. This results in the fewest number of questions in each country, rather than a standard set of questions across countries. On average, EquityTool questionnaires are 66% shorter than the original DHS survey.
  2. Maintain a minimum level of agreement with the original DHS wealth index. We use a measure of reliability called the kappa statistic (k) to compare the short index to the original DHS wealth index, and insist that k>= 0.75 for all indices.
  3. Allow for reliable sub-group analyses in urban areas. Each index is equally reliable if comparing respondents to the national or urban only wealth distributions.

How is the alternative wealth index created?

To create an alternative wealth index for a specific country, the recent DHS data and wealth index files for a country are downloaded.  If recent DHS data is not available for a country, other nationally representative surveys may be appropriate.

Through our pilot testing and feedback process, several questions were found difficult to answer in the contexts for which this survey is designed (outside of the home). These questions are removed. Then, questions are ordered by statistical importance to the full wealth index, and added to the questionnaire in order, until a sufficient level of agreement is reached between the full wealth index and our simplified index. We ensure that respondents are sorted into the same wealth quintiles by both indices at a level of agreement where kappa >= 0.75.  This level of agreement must be achieved when comparing the simplified index to the full index for the national and urban only populations. Occasionally, this method does not produce a reliable index.  Any changes to the standard process are described in the country factsheet, available on our Countries page.

 

For more information on how the alternative wealth index was chosen and created, see our full article in Global Health: Science and Practice.

Metrics for Management (M4M) has translated this methodology into the EquityTool, which can now be used in three key ways. Data can be collected through a user-friendly web based application, or the questionnaire can be added to an existing survey or integrated into DHIS2.

The EquityTool is free to use and does not require training.

Discover the right data collection platform to fit your needs here.