Author: Nursing-research

I am professor, emeritus, of nursing at Texas Woman's University, Denton, Texas

Inspired

Below you will find the note I sent to Wendy Davis’ organization, Deeds Not Words.  It is my first move toward engaging with activist groups to assist with Citizen Science.

 

To Wendy Davis at Deeds Not Words

I attended the Circle of Health event at the Texas Theater last night and was struck by the fact that the women on the panel were voicing the need for “evidence-based” care. As a nurse and researcher, searching for evidence is at the forefront of what I do. However, I realize that researchers’ own implicit bias and ignorance of the needs of women, especially those women of color, affect and the research they do to produce much of the “evidence” surrounding maternal mortality.

One case in point is the focus of the Texas Task Force on Maternal Morbidity and Mortality on the measurement of maternal mortality. The political act of measuring maternal mortality has taken center stage and even found its way into mainstream Obstetric medicine. The new “enhanced” method of measuring mortality only in the first 42 days following pregnancy and only deaths “related to pregnancy”, significantly narrowed the view of the problem, making maternal mortality a problem that can be solved by improving hospital care rather than improving the systemic bias and social injustices which so heavily influence the risk of a mother’s death. For example, the Task Force has issued no recommendations for addressing maternal mortality caused by assault even though homicide is one of the leading causes of death of women who are pregnant or have been pregnant within one year of their death.

As Marsha Jones pointed out, the researchers and clinicians serving on the task force are talking among themselves rather than with the people affected. Professionals learn primarily from their peers and see the problem through professional lenses. Doctors and nurses view the problem as a technical one which can be solved by adhering to treatment “bundles”. Economists and policy makers view the problem as one of scarce or maldistributed resources. Certainly, those views of the problem can lead to needed action but until the voices of women and their families are heard, critical knowledge of the full scope of challenges women of color face every day will remain hidden from the view of those who can ease those challenges and reduce the risk they will lead to death.

My colleagues and I at TWU have been studying infant mortality and maternal mortality for more than 20 years. Our research team is made up of nurses who have expertise in public health, midwifery, ob/gyn nursing and intimate partner violence. After 20 years we have learned a great deal about the scope of infant mortality and maternal mortality and the factors related to them. What we have not learned, however, is how to remove the systemic biases that create and sustain factors contributing to these preventable events.

I believe that President Obama’s idea of Citizen Science is the key to finding the missing evidence needed to solve problems of infant mortality and maternal mortality. Paraphrasing his words, “Science should be about all people, for all people, and by all people.” In that spirit I would like to offer whatever research skills our team can provide to groups such as those we heard on last night’s panel. Women of color are noticeably absent from the Task Force on Maternal Morbidity and Mortality and from the academic settings in which much research is carried out. Likewise, researchers are noticeably absent from activist organizations which have the will and energy to put evidence into practice. I am reaching out to Deeds Not Words to address at least one half of this deficit.

Please contact me if you see a need our team could meet.

Patti Hamilton, RN, PhD
Emeritus Professor
College of Nursing
Texas Woman’s University
940 759 2055
phamilton@twu.edu

Never too old to learn something new…..

old_dog

RAND Appropriateness Method (RAM)

I love finding something that is new to me about a topic I thought I was well informed about.  That happened recently while working on a project to create guidelines for publications about missed, rationed, or unfinished nursing care. Our work group is made up of two nurses from Switzerland, one from Italy, and me.

The graduate student from Switzerland has been responsible for setting up a Delphi-like process to gain information about what which elements of a research article are particularly important to include when reporting research results about missed, rationed, or unfinished nursing care. She settled on the Rand/UCLA Appropriateness Method (RAM).

Even though for my own dissertation in 1988 I used the Delphi method (which was also developed by RAND) I had never heard of the RAM. So, I searched for background and explanation of the method.  I also checked to see what publications in nursing had reported using the RAM.  I found very few examples of RAM being used to evaluate nursing interventions or policies. See the bottom of this post for an example that might be of interest for nurses, however.

The overview below came directly from the RAND/UCLA Appropriateness Users’ Manual.

With so few randomized controlled trials (termed the “gold standard” of research) on which to base decisions about appropriateness of nursing actions, I believe the RAM has a promising role to play in evidence-based decision making in nursing.

I would love to hear from you about your thoughts on the method and its role in nursing or in your own field.

An Overview of the Method The basic steps in applying the RAM are shown in Figure 1. First, a detailed literature review is performed to synthesise the latest available scientific evidence on the procedure to be rated. At the same time, a list of specific clinical scenarios or “indications” is produced in the form of a matrix which categorises patients who might present for the procedure in question in terms of their symptoms, past medical history and the results of relevant diagnostic tests. These indications are grouped into “chapters” based on the primary presenting symptom leading to a patient’s being referred for treatment or considered for a particular procedure.

 Figure 1: The RAND/UCLA Appropriateness Method

 An example of a specific indication for coronary revascularization in the chapter on “Chronic Stable Angina” is: A patient with severe angina (class III/IV) in spite of optimal medical therapy, who has 2-vessel disease without involvement of the proximal left anterior descending artery, an ejection fraction of between 30 and 50%, a very positive stress test, and who is at low to moderate surgical risk.

 A panel of experts is identified, often based on recommendations from the relevant medical societies. The literature review and the list of indications, together with a list of definitions for all terms used in the indications list, are sent to the members of this panel.

For each indication, the panel members rate the benefit-to-harm ratio of the procedure on a scale of 1 to 9, where 1 means that the expected harms greatly outweigh the expected benefits, and 9 means that the expected benefits greatly outweigh the expected harms. A middle rating of 5 can mean either that the harms and benefits are about equal or that the rater cannot make the judgement for the patient described in the indication. The panellists rate each of the indications twice, in a two-round “modified Delphi” process. In the first round, the ratings are made individually at home, with no interaction among panellists.

In the second round, the panel members meet for 1-2 days under the leadership of a moderator experienced in using the List of indications and definitions

 

Criteria:

  • Appropriate
  • Uncertain
  • Inappropriate

 

% of use that is:

  • Appropriate
  • Uncertain
  • Inappropriate

Retrospective: Comparison with clinical records 1st round: no interaction 2nd round: panel meeting Prospective: Clinical decision aids Increase appropriateness Literature review and synthesis of the evidence Expert panel rates indications in two rounds page 5 method. Each panellist receives an individualised document showing the distribution of all the experts’ first round ratings, together with his/her own specific ratings. During the meeting, panellists discuss the ratings, focusing on areas of disagreement, and are given the opportunity to modify the original list of indications and/or definitions, if desired.

 After discussing each chapter of the list of indications, they re-rate each indication individually. No attempt is made to force the panel to consensus. Instead, the two-round process is designed to sort out whether discrepant ratings are due to real clinical disagreement over the use of the procedure (“real” disagreement) or to fatigue or misunderstanding (“artifactual” disagreement).

Finally, each indication is classified as “appropriate,” “uncertain” or “inappropriate” for the procedure under review in accordance with the panellists’ median score and the level of disagreement among the panellists. Indications with median scores in the 1-3 range are classified as inappropriate, those in the 4-6 range as uncertain, and those in the 7-9 range as appropriate. However, all indications rated “with disagreement,” whatever the median, are classified as uncertain. “Disagreement” here basically means a lack of consensus, either because there is polarisation of the group or because judgements are spread over the entire 1 to 9 rating scale.

As discussed in Chapter 8, various alternative definitions for disagreement have been used throughout the history of the RAM. Appropriateness studies sometimes categorise levels of agreement further to identify indications rated “with agreement” and those rated with “indeterminate” agreement (neither agreement nor disagreement). Depending on how the appropriateness criteria are to be used, it may sometimes be desirable to identify those indications rated with greater or lesser levels of agreement. If necessity criteria are also to be developed, a third round of ratings takes place, usually by mail, in which panellists are asked to rate the necessity of those indications that have been classified as appropriate by the panel. The RAM definition of necessity (Kahan et al., 1994a) is that:

  • The procedure is appropriate, i.e., the health benefits exceed the risks by a sufficient margin to make it worth doing.
  • It would be improper care not to offer the procedure to a patient.
  • There is a reasonable chance that the procedure will benefit the patient.
  • The magnitude of the expected benefit is not small.

All four of the preceding criteria must be met for a procedure to be considered as necessary for a particular indication. To determine necessity, indications rated appropriate by the panel are presented for a further rating of necessity. This rating is also done on a scale of 1 to 9, where 1 means the procedure is clearly not necessary and 9 means it clearly is necessary. If panellists disagree in their necessity ratings or if the median is less than 7, then the indication is judged as “appropriate but not necessary.” Only appropriate indications with a necessity rating of 7 or more without disagreement are judged “necessary.” Comparison with Other Group Judgement Methods

The RAM is only one of several methods that have been developed to identify the collective opinion of experts (Fink et al., 1984). Although it is often called a “consensus method,” it does not really belong in that category, because its objective is to detect when the experts agree, rather than to obtain a consensus among them. It is based on the so-called “Delphi method,” developed at RAND in the 1950s as a tool to predict the future, which was applied to political-military, technological and economic topics (Linstone et al., 1975).

The Delphi process has since also come to be used in a variety of health and medical settings. The method generally involves multiple rounds, in which a questionnaire is sent to a group of experts who answer the questions anonymously. The results of the survey are then tabulated and reported back to the group, and each person is asked to answer the questionnaire again. This iterative process continues until there is a convergence of opinion on the subject or no further substantial changes in the replies are elicited.

The RAM is sometimes miscast as an example of the Nominal Group Technique (NGT). NGT is a highly structured process in which participants are brought together and asked to write down all their ideas on a particular subject. The moderator asks each person to briefly describe the most important idea on his or her list, and continues around the table until everyone’s ideas have been listed. After discussion of each topic, participants are asked to individually rank order or rate their judgement of the item’s importance on a numerical scale. Different mathematical techniques are used to aggregate the results. The RAM, unlike the NGT, begins with a highly structured list of clinical indications, and the discussion is tightly linked to the basic measurement of appropriateness.

A third group judgement method is the Consensus Development Conference. The U.S. National Institutes of Health (NIH) have a mandate to evaluate and disseminate information about health care technologies and biomedical research (Kanouse, 1989). To this end, they have developed what are known as NIH Consensus Conferences, which bring together a wide variety of participants, including physicians, researchers and consumers, who are charged with developing a mutually acceptable consensus statement to answer specific, pre-defined questions about the topic. This process includes conducting a literature review, summarising the current state of knowledge, presentations by experts and advocates, and audience discussion. These conferences frequently last 2 or more days, and do not end until the participants have agreed on a written statement.

Many European countries have developed their own versions of Consensus Conferences. At its centre, the RAM is a modified Delphi method that, unlike the original Delphi, provides panellists with the opportunity to discuss their judgements between the rating rounds. Contrary to the fears of the original developers of Delphi, experience with the RAM and the contemporaneous literature on group processes both indicate that the potential for bias in a face-to-face group can be largely controlled by effective group leadership (e.g., Kahan et al., 1994b). Thus, while panellists receive feedback on the group’s responses, as is done in the classic Delphi method, they have a chance to discuss their answers in a face-to-face meeting, similar to the NGT and NIH Consensus Conferences

The following article used a modified RAM and illustrates the application of the method to quality of life.

Improving Methods for Measuring Quality of Care: A Patient-Centered Approach in Chronic Disease Barbara G. Bokhour, Mary Jo Pugh, Jaya K. Rao, Ruzan Avetisyan, Dan R. Berlowitz, and Lewis E. Kazis Medical Care Research and Review  Vol 66, Issue 2, pp. 147 – 166