I had intended my next post to be about Thinking Outside the Box but there has been an argument brewing in the news and in academic journals that I think is worth mentioning first.
The argument started in 2016 when Marian MacDorman and her colleagues published findings showing an exceptionally high maternal mortality rate in Texas. MacDorman’s paper was picked up by public media including National Public Radio and the Dallas Morning News and the startling news spread across the state and the U.S. In 2013 the Texas Legislature had established a Task Force to monitor maternal mortality and MacDorman’s report focused increased attention on their efforts. In 2016 the Task Force issued its biennial report showing that most maternal mortality in 2011-2012 had occurred more than 42 days following the end of pregnancy and that drug overdose, homicide and suicide were also recorded as underlying causes of maternal mortality. The Texas Legislature, in 2017, expanded the scope of the Task Force to include monitoring maternal morbidity as well as mortality. As often happens, the expanded scope did not come with commensurate funding.
In April, 2018, a group of authors from the Task Force published a paper in the journal, Obstetrics and Gynecology, in which they challenged MacDorman’s method of analysis and claimed to have developed an “enhanced” method for studying maternal mortality. Below I will give you my view of the issues at stake and will explain why I do not believe the Task Force’s method should be considered an “enhanced” method of measuring the extent of risk and incidence of maternal death. Instead, it should be considered a method which focuses Task Force efforts on understanding what occurred to result in mortality in a specific subset of cases. Their new method limits the definition of maternal mortality to those deaths within the first 42 days following the end of pregnancy and does not include causes of death in such women from overdose, homicide, or suicide. The Centers for Disease Control (CDC) states that codes referring to maternal death include not only the first 42 days following the end of pregnancy, but also the later period lasting from 43 to 365 days following pregnancy. The Task Force method is hospital-centric in that it appears to focus on deaths from physiological complications which can be managed in hospital settings during the early postpartum period ignoring the more intractable causes of death that women face after the postpartum period ends. I am not calling the method unsound methodologically speaking. Rather, I find it lacks validity in usefulness for comprehensively identifying and addressing the roots of maternal mortality in Texas.
The implications of the suggested “enhanced” method with its suggested omissions are great. The method:
- Focuses only on typical co-morbid conditions treatable during and immediately following pregnancy
- Omits the later part of the postpartum period when access to services are often less accessible than during pregnancy and the first 42 days following the end of pregnancy
- Ignores important evidence-based causes of maternal mortality such as drug use, homicide, and suicide
- Uses a single year’s data to justify its claim of improvement over other methods of measuring maternal mortality
- Provides little, if any, direction for developing interventions that are innovative and reach beyond hospital walls.
What is Research Validity?
I want to be transparent in my reasoning so I will share what I have long believed to be true about judging the validity of research.
Years ago I used a book titled Validity and the Research Process by Brinberg and McGrath (1986) to teach the final synthesis course in TWU’s Nursing PhD program . That one book addressed issues of construct validity, measurement reliability and validity, external and internal validity, and real-world issues involved in conducting research and applying the resulting findings. The approach the authors took was not one of simply describing how to understand key research principles but, rather, how to link the positive and negative results involved in combining these principles so that the results were valid. That book is a treasure that I still find applicable all these years later.
My discussion in this post involves, primarily, Brinberg and McGrath’s first stage of research planning and evaluation which they call Validity as Value. By value they mean the worth, usefulness, or importance of the research. Rigorously carried out research is only truly valid if it yields value.
Validity as Value in the Preparatory Stage
In stage 1 the researcher should focus on finding:
- the most valuable events and phenomenon for study
- the most valuable method for collection, analysis of data
- the most valuable concepts and explanations that interpret the observations
Keeping Validity as Value in mind, I want to acquaint you with an ongoing clinically, academically, and philosophically contested study of the rate of maternal mortality in Texas and the U.S. This isn’t only a research issue; the debate over just how bad the Texas maternal death rate is will likely be translated into state and even national funding decisions, voting behaviors, and, more importantly, into the lives of Texas women and their families from all walks of life.
The news paper article below appeared immediately following a scientific article appearing in the May issue of Obstetrics and Gynecology and written by members of the Texas Maternal Mortality Task Force along with officials and staff of the Texas Department of State Health Services (DSHS). The public in Texas will read the news paper article below and interpret its message in various ways in light of their own opinions, experiences, and beliefs about research.
My experiences and beliefs about the May 2018 article by the Texas Task Force and the popular press articles reporting it derive from experiences first as a Public Health Nurse in Fort Worth, from working with similar data for over 20 years, and from my belief that the validity of research findings should first be measured against the principles in Brinberg and McGrath’s Stage 1 guidelines.
Let’s start with the first consideration of Brinberg and McGrath’s values in Stage 1 research: the most valuable events and phenomenon for study. The Task Force obtained 147 death records from CDC’s National Center for Health Statistics for analysis. These records were selected because they represented deaths among Texas residents whose death record indicated the underlying causes of death was related to obstetric causes within one year of the end of pregnancy in 2012. There is no question that the Task Force members realized the significance of even one death to a woman during or within one year of pregnancy and the value of knowing the extent of this problem in Texas.
The DSHS website states: Maternal Mortality and Morbidity Task Force was created by Senate Bill 495, 83rd Legislature, Regular Session, 2013. The multidisciplinary task force within the Department of State Health Services (DSHS) will study maternal mortality and morbidity. The task force will study and review cases of pregnancy-related deaths and trends in severe maternal morbidity, determine the feasibility of the task force studying cases of severe maternal morbidity, and make recommendations to help reduce the incidence of pregnancy-related deaths and severe maternal morbidity in Texas.…
The World Health Organization (WHO) has stated “Maternal mortality is a health indicator which shows very wide gaps between rich and poor, urban and rural areas, both between countries and within them…” The Texas Legislature, which approved and funded the Task Force, the Task Force Members, popular media such as the Texas Tribune and National Public Radio, and the WHO would seem all to be in agreement that studying maternal mortality passes the test for VALUE in studying these events.
Next, let’s consider the most valuable phenomenon to consider when studying maternal mortality. The May Task Force paper states that the phenomenon for study was operationalized in such a way as to reduce the original 147 cases to a final 56. “For all 147 obstetric-coded deaths, a woman was considered to have a confirmed maternal death while pregnant or within 42 days postpartum 1) if her death record could be matched with a live birth or fetal death occurring within 42 days of the date of death; 2) if her medical records, autopsy or other death investigation records, or information received from contacting the death certifier indicated either pregnancy at the time of death or pregnancy within 42 days of the date of death; or 3) to err on the side of caution, if the death certificate narrative indicated pregnancy at time of death or within 42 days of the date of death when sufficient medical records were not received.”(p.764)
Death records are coded using an international classification of disease (ICD) system. That system was updated to version 10 in 2003. The CDC uses the following definitions when defining maternal deaths.
The Task Force chose to focus on deaths within a period of 42 days following the end of pregnancy. They may have chosen that period because of the greater likelihood that medical records and other death investigation results would be more likely to have included correct information on the time between pregnancy and death than if the death occurred later than 42 weeks after the termination of pregnancy (from birth, miscarriage, or abortion from any cause). However, here is one of the validity issues Brinberg and McGrath wrote about, the trade-off between precision and realism.
In 2016 the Task Force wrote in their biennial report that the majority of maternal deaths in 2011-2012 occur later than 42 days after the end of pregnancy. Below is the figure reporting these results in the 2016 report. One of their findings stated, “A majority of maternal deaths occur later than 42 days after delivery Maternal deaths were confirmed by linking each mother’s information to a birth or fetal death event occurring up to 365 days prior. Time between the two events was calculated in days and a survival plot was generated to help visualize the relationship between maternal mortality and time. Figure 3 shows the percent of women in the 2011-2012 maternal death cohort who remained alive at particular points in time over the 365 days following their delivery.”(p.7)
Note: The data described in the figure above refer to 2011 and 2012 while the May paper describes only 2012 data.
They go on to explain:
“The WHO defines all maternal deaths within 42 days of the end of pregnancy as pregnancy-related deaths, irrespective of the cause of death (WHO, 2016). However, close to 60 percent of maternal deaths in 2011-2012 occurred after 42 days post-delivery (Figure 3). Case review of these deaths will determine whether they were pregnancy related,-associated, or neither. Nevertheless, it is clear that women remain at-risk for the first year after their pregnancy has ended. It is possible that lack of continuity of care plays a role in these later maternal death outcomes.” (p. 7)
Next, I call your attention to the Stage 1 Vaidity as Value principle: most valuable method for collection, analysis of data.
The Task Force reported in 2016 that the death records were matched with birth or infant death records. I have found no evidence that the Task Force’s “confirmation by linking” protocol described in their report was considered to be incorrect. Could the decision to omit late period maternal deaths wassed on some difference in the data from 2011 and 2012? In the absence of a clear explanation I am forced to conclude that the decision to focus only on deaths within 42 days was based entirely on the desire to be precise in identifying “true” maternal deaths. They mention that in their two-step method to find cases they searched birth records and found six additional maternal deaths not noted in death records but they omitted those six because death had occurred 43 or more days postpartum. They state in their May paper, “Maternal deaths that occurred 43 or more days postpartum were also excluded from analyses because these deaths occurred after the time frame of interest.” (p.764)
Finding no explicit explanation beyond a goal of precision, I must conclude precision took precedence over realism and the decision concerning which phenomenon to be studied was based more strongly on seeking an irrefutable method than on need within the population. That being the case, I believe it is vitally important to speak up and point out that statements such as the one appearing in the Texas Tribune article, ” ..the number of women who died dropped from 147 to 56.” is grossly misleading. Those 147 women did die. They were not counted by the Task Force simply because their death records did not satisfy criteria to be considered a maternal mortality according to the Texas Task Force definition and time frame. Thus, value of the methodology and analysis chosen by the Task Force should be measured in terms of precision rather than in terms of learning about a full spectrum of causes and preventive strategies where maternal mortality is concerned. We might call this trade-off one of specificity over sensitivity.
The Task Force’s focus on precision and specificity is understandable in light of its charge by the legislature to “…determine the feasibility of the task force studying cases of severe maternal morbidity. “ It has taken the Task Force tremendous effort, commitment, and more than 60 months working with limited resources to complete its analysis of maternal mortality in Texas in 2012. How feasible is it to continue to carry out such costly and difficult analysis in the future? Certainly it is much more feasible to analyze 56 deaths than 147. Doesn’t Texas need a system that does not lag behind more than 5 years in its monitoring of this critical public health indicator ?
Uneven quality of information found in death records were reported in the May paper. For example, the authors wrote: “One hundred of the 147 obstetric-coded deaths were unable to be confirmed as maternal deaths. Eleven of the 147 obstetric-coded deaths (7.5%) occurred 43 or more days postpartum (outside of the 42-day time frame of interest), 74 (50.3%) showed no evidence of pregnancy after data matching and record review, and 15 (10.2%) had insufficient information to make a determination…”(p.765)
Certainly, the quality of available data can be at fault in any over-estimation of the number of maternal deaths, but as the May paper reports, when maternal deaths were found outside of 42 days postpartum, they were not included. Also, the Task Force did not mention the equally troubling problems with accuracy of birth records used to confirm maternal deaths. Over my 20 years of working with birth certificate data I have found them to have serious deficits in quality and completeness. (for more information on birth record quality see these two dissertations conducted at Texas Woman’s University: Polancich, S. (2002). Birth certificate data quality : Examining the maternal medical risk factor text field … and Restrepo, E. (2002) Birth certificate data quality: a study of medical risk…
Isn’t it likely that errors in birth records resulted in failure to confirm by linking birth and death records and under-estimation of maternal mortality?
Finally, I want to comment on Brinberg and McGrath’s third principle of Validity as Value, namely: the most valuable concepts and explanations that interpret the observations. The May paper by the task force uses the term “enhanced method” when describing their analysis of maternal mortality. Webster’s dictionary defines enhance as heighten, increase; especially : to increase or improve in value, quality, desirability, or attractiveness. I would agree that their method removed some erroneous cases of maternal mortality. I cannot agree that their method of analyzing maternal mortality for the purpose of preventing it has been improved as a result of those removals. The omission of mortality after 42 days postpartum and using only obstetric codes for the underlying causes of death and omitting violence, suicide, and drug overdose mask other significant sources of mortality. The Task Force’s 2016 report included the following figure and explanation:
For this examination of the statewide trends, maternal deaths were identified by linking the mothers’ information from their birth/fetal death record with information from their own death record occurring within one year of pregnancy termination, regardless of the ICD-10 code assigned. Some alternative methodologies do begin by identifying all obstetric deaths, labeled with an “O-code” (ICD-10 code O00-O99). However, this approach likely fails to identify maternal deaths with non-natural causes, such as overdose, suicide, and homicide. Clearly, drug overdose, homicide, and suicide are important causes of maternal mortality not included in the “enhanced” method presented in the May paper.
I do not believe the May paper presents a convincing argument for calling their method “enhanced”. The authors concluded “…a method enhanced with data matching and review of relevant medical and death investigation yields more accurate ratios.” (p. 769) True, but omitting previously identified significant causes of death from overdose, homicide, and suicide and limiting the time frame for death to 42 days or less substantially diminishes the amount of information needed to reduce overall maternal mortality. I would have less trouble if their claim was not a more accurate method but rather was a method for focusing on a verified subset of deaths for closer scrutiny.
The Task Force’s interpretation of their results rests on analysis of a single year of data (2012). Those same data were analyzed along with data from 2011 and results included in their biennial report. The Task Force emphasized the importance of deaths after 42 days and the prevalence of homicide, suicide and drug overdose.
In their discussion in their May paper the authors state their method should be interpreted as finding a large number of miscoded deaths. (p. 768) Certainly, there were miscoded death records but there also was a purposeful omission of legitimate death records not conforming to their inclusion criteria. Such subtle points are often overlooked and overstated as in the Texas Tribune’s article on April 9th in which they state the number of maternal deaths were much fewer.
Maternal mortality has become a contentious issue. One reason is because it has been suggested (without research evidence to support the claim) that the sharp increase in Texas Maternal Mortality coincided with Texas’ cutbacks on Medicaid services and reductions in funding for Planned Parenthood.
Marian MacDorman and her colleagues first reported exceptionally high rates of maternal mortality in Texas in 2016. See Macdorman, M., Declercq, E., Cabral, H., & Morton, C. (2016). Recent Increases in the U.S. Maternal Mortality Rate: Disentangling Trends From Measurement Issues. Obstetrics and Gynecology, 128(3), 447-55. In this paper they make a statement in their discussion that , “There were some changes in the provision of women’s health services in Texan from 2011-2015, including the closing of several women’s health clinics.” (p. 454) They do not mention Planned Parenthood specifically. Nevertheless, researchers supported by an organization with a strong anti-abortion agenda picked out that one statement and have challenged the findings of MacDorman and her team in the same May, 2018, issue of Obstetrics and Gynecology in which the latest paper of the Task Force appears. The Letter to the editor challenging the MacDorman team’s research is from James Studnicki and John Fisher, of the Charlotte Lozier Institute. They write: “Two important research and public policy issues underscore our request for a formal correction or clarification of these findings. First, the alleged 2010–2012 doubling, juxtaposed with the closing of women’s health clinics around that time, enabled a conventional wisdom to emerge and to be widely disseminated in the popular media that the closing of Planned Parenthood clinics in Texas was causing maternal deaths. In scientific terms, no such cause and effect has ever been demonstrated, nor does the MacDorman article provide the temporal sequence to suggest that relationship. “(p 934)
Value in research is difficult to obtain because demands for rigor and precision must be balanced with the messiness and ambiguity of the focal problem as it exists in the real world. The Texas Task Force on Maternal Mortality and Morbidity is faced with an almost impossible charge to accurately assess the extent and causes of maternal mortality along with their new obligation to track severe maternal morbidity. They have experimented with a method that is feasible and reproducible. The cost of that feasibility is the omission of additional causes of maternal death later in the year following pregnancy.
The implications of the suggested “enhanced” method with its suggested omissions are great. The method:
- focuses only on typical co-morbid conditions treatable during and immediately following pregnancy;
- omits the later part of the postpartum period when access to services are often less accessible than during pregnancy and the first 42 days following the end of pregnancy;
- ignores evidence-based causes of maternal mortality such as drug use, homicide, and suicide;
- uses a single year’s data to claim its improvement over other methods of measuring maternal mortality;
- provides little, if any, direction for developing interventions that are innovative and reach beyond hospital walls;
- is extremely costly and time consuming with lag times of 4-5 years after deaths occurred; and and these limitations result in missed opportunities to learn what to do and what is working to reduce maternal mortality.
In the final instance, it is no less tragic if 56 women die following pregnancy than if 147 died. As the WHO states, “Most maternal deaths are preventable, as the health-care solutions to prevent or manage complications are well known. All women need access to antenatal care in pregnancy, skilled care during childbirth, and care and support in the weeks after childbirth”. No woman should die in Texas or elsewhere from a preventable cause.