Notes on Methodology

The California Women’s Well-Being Index is a county-level, composite index that shows how women are faring across California. The Index consists of 30 indicators of women’s well-being that are grouped into five dimensions: Health, Personal Safety, Employment & Earnings, Economic Security, and Political Empowerment. The methodology used to create the index is outlined in detail below.

Creating the Women’s Well-Being Index

In deciding which measures, or “indicators,” to include in the Index, the Budget Center first engaged in a thorough review of research on women and families as well as an evaluation of comparable projects at the national and state levels.[1] This review resulted in a wide-ranging list of potential indicators. Because the Index measures women’s well-being, potential indicators were limited to “outputs” – that is, those that measure attributes of the female population or reflect community characteristics. This standard eliminated “input” indicators, such as those that measure public spending or community resources, for example.

The Budget Center subsequently screened data sources to ensure that data for potential indicators were current, available from a reputable source, and based on sound research methods. Data also were screened to verify that they are updated on a regular basis in order to allow for updates to the WWBI in future years. This initial search resulted in a list of 50 potential indicators.

Next, the Budget Center, in cooperation with the Women’s Foundation of California, surveyed individuals who focus on issues of concern to women across the state. (See the Acknowledgments for a list of survey respondents.) The survey allowed respondents to rank the importance of potential indicators in measuring women’s well-being as well as to suggest alternative indicators. The initial survey was sent via email to approximately 200 individuals. Three follow-up emails were sent to the same individuals, yielding a response rate of approximately 30 percent. Based primarily on survey results, the initial list of 50 indicators was narrowed down to 30 indicators falling within five dimensions of women’s well-being.

Data Sources and Data Quality

The 30 indicators included in the Women’s Well-Being Index are based on data from a variety of state and federal agencies as well as from universities. Data for 18 indicators come from surveys, including the US Census Bureau’s American Community Survey (ACS) and the California Health Interview Survey (CHIS) conducted by University of California, Los Angeles. The remaining data come from administrative sources, with this information collected by various agencies and organizations, such as the California Department of Justice or the California Department of Public Health. Data for most indicators reflect multiyear estimates. (Combining data from multiple years increases the reliability of the data.) These multiyear estimates reflect the average condition in each county during a specific time period.

For survey data, margins of error at the 95 percent confidence level are included in the Index whenever possible and applicable.[2] In addition, margins of error at the 95 percent confidence level are included for some administrative data. While these data are not subject to sampling error – because they reflect a full “universe” of individuals rather than a sample – it is common to provide the margin of error for certain vital statistics because they are subject to random variation during any given period.

In some cases, due to data limitations, data for certain counties for specific indicators were deemed unreliable. The Budget Center used several common benchmarks to determine if a data point was unreliable. This includes an event count of less than 20 in a county for any given time period and/or a coefficient of variation greater than or equal to 30 percent. For the ACS data, a higher standard was applied: a coefficient of variation greater than or equal to 10 percent and, for data reported as percentages, a margin of error greater than or equal to 15 percent.

When data for one or more counties were determined to be unreliable, county groups were created by aggregating estimates. This was done in order to create a more stable estimate applicable to each county in the group. When creating county groups, several considerations were made. First, contiguous counties with unreliable estimates were often grouped together. However, if possible, county groups were created with the smallest number of counties to avoid unnecessary loss of detail. For example, if the data for four contiguous counties were deemed unreliable, ideally two county groups would be created instead of one large county group. In some cases, data for one or more counties were grouped with a county that had a reliable estimate in order to create a stable estimate.

In addition, indicator values were considered when aggregating counties. For example, if a county with an unreliable estimate needed to be grouped with another county, the indicator values were taken into account in order to avoid aggregating counties with disparate estimates. In some cases, groups do not consist of contiguous counties because the estimates for the contiguous counties were too different to be combined.

Calculating the Women’s Well-Being Index

The Index consists of a wide range of data reported in a variety of ways, such as rates, ratios, and percentages. These data have varying ranges and scales. In order to construct a composite index, indicators within each dimension were standardized and aggregated to create a county-level value for each dimension and for the overall index. This section outlines the methodology used in standardizing and aggregating the data in the Women’s Well-Being Index.

Data were standardized for each indicator by calculating the z-score. The z-score converts a value into units of measurement based on the standard deviation. The z-score is calculated using the 58-county averages and standard deviations for each indicator. This allows for comparing values across indicators with different formats and with varying ranges of data. The z-score is calculated for each county and indicator using the following formula:

Methodology equation

 

 

 

Formula part 2

In some instances, a higher z-score indicates greater well-being, such as the percentage women with at least a high school diploma. In other cases, a higher z-score indicates lesser well-being, such as the female unemployment rate. In order to ensure that higher scores on the Index consistently reflect greater well-being, a number of indicators were reverse-coded. This was done by multiplying the z-score by negative 1.

One disadvantage of using z-scores is that the value of the z-score is hard to interpret. In order to create a value that is easy to understand, the z-scores for each indicator were converted into a 100-point scale using the following formula:

Methodology equation 2

 

 

 

The highest scaled z-score for any given indicator has a value of 100, and the lowest scaled z-score has a value of 0, with higher scores indicating greater well-being. However, while a score of 100 reflects the best value across all counties, it is not indicative of maximum well-being. For example, a score of 100 for the voter registration indicator does not mean that all eligible women in that county are registered to vote. Likewise, a score of 0 does not indicate that no women in that county are registered to vote.

To calculate scores for each of the five dimensions, we averaged the scaled z-scores for each county within each dimension. We then calculated overall Index scores by averaging each county’s five dimension scores. The indicators and dimensions were not weighted prior to aggregation (i.e., they have equal weights). This reflects the belief that each indicator within a dimension has equal bearing on the well-being of women. Likewise, equally weighting the five dimensions indicates that health, personal safety, employment and earnings, economic security, and political empowerment are all equally important in assessing how women are faring in California.[3]

Counties are ranked by indicator score, by dimension score, and by their overall score on the Index. In general, the Index employs a “modified competition ranking system” to rank the counties. In a modified competition ranking system, ties are ranked with the lowest rank. For example, if three of California’s 58 counties are tied for last, they would have a rank of 58. Using the traditional ranking system, their tied rank would be 56. The exception to this rule is when two or more counties are tied for first. When this occurs, these counties are ranked #1.

[1] See for example, Helen Boutrous, et al., The Report on the Status of Women and Girls in California: 2015 (Mount Saint Mary’s University, Los Angeles: 2015); Anna Chu and Charles Posner, The State of Women in America: A 50-State Analysis of How Women Are Faring Across the Nation (Center for American Progress: September 2013); Cynthia Hess, et al., The Status of Women in the States 2015 (Institute for Women’s Policy Research: May 2015); Kristen Lewis and Sarah Burd-Sharps, Women’s Well-Being: Ranking America’s Top 25 Metro Areas (Measure of America: April 2012); University of Minnesota, Center on Women and Public Policy, and Women’s Foundation of Minnesota, Status of Women & Girls in Minnesota: Research Overview (June 2014); Wider Opportunities for Women, The Economic Security Scorecard: Policy and Security in the States (2013).

[2] A 95 percent confidence level means that a researcher is 95 percent confident that the interval defined by the margins of error contains the true value for the population as a whole.

[3] For more details on weighting within composite indexes and other methodological issues, see Organization for Economic Co-Operation and Development, Handbook on Constructing Composite Indicators: Methodology and User Guide (2008).