|Year : 2018 | Volume
| Issue : 2 | Page : 83-89
Development and validation of saving mothers score: A comprehensive scoring system for early identification of sick mothers
Kousalya Chakravarthy1, Sunil T Pandya2, Praveen K Nirmalan3
1 Department of Anaesthesia, MGMH Petlaburj, Osmania Medical College, Hyderabad, Telangana, India
2 Department of Anaesthesia, Fernandez Hospital, Hyderabad, Telangana, India
3 Manasvin's Centre for Marital and Family Therapy, Hyderabad, Telangana, India
|Date of Web Publication||3-Oct-2018|
Dr. Kousalya Chakravarthy
3-5-1083, Flat 306, Sri Tara Jeet Residency, Narayanaguda, Hyderabad - 500 029, Telangana
Source of Support: None, Conflict of Interest: None
Context: Early identification of high-risk parturient and maternal physiological deterioration may reduce maternal morbidity and mortality. Aims: This study aimed to develop a comprehensive scoring system 'Saving Mothers Score' (SMS) to identify the sick mothers and validate SMS against the existing Modified Early Obstetric Warning System (MEOWS). Settings and Design: The SMS was developed through a formative research, item pool generation, content and construct validity. Methodology: Thirty-three (33) items were identified and pooled into three parameters pregnancy-related risk factors, physiological variables and biochemical tests. Each item was given a colour coding and a score. The trigger and score of SMS was prospectively analysed in 120 obstetric in-patients. Statistical Analysis Used: Tests of diagnostic effectiveness and 95% confidence intervals (95% CI) around point estimates. Results: Forty six women triggered (38.33%; 95% CI: 29.96, 47.26) and 41 (81.93%, 95% CI: 77.54, 95.91) of these 46 women developed morbidity. The overall accuracy of SMS chart was similar for trigger [sensitivity 60.9%; specificity 98.6%, area under receiver operator characteristic curve (AUROC) 0.80] and scoring (sensitivity 56.1%; specificity 92.4%, AUROC 0.74) with positive predictive value (PPV) and negative predictive value (NPV) of 96.6% and 80.2%, respectively. The accuracy of SMS was comparable to MEOWS (sensitivity 54.6%, specificity 97.8%, PPV 92.5% and NPV 79.9%). Conclusions: The diagnostic effectiveness of SMS was comparable to MEOWS. SMS may be used as a screening test to identify a sick mother. SMS can predict morbidity, help in triage and early intervention or timely referral to a higher centre.
Keywords: High-risk pregnancy, maternal mortality, sensitivity, specificity, trigger
|How to cite this article:|
Chakravarthy K, Pandya ST, Nirmalan PK. Development and validation of saving mothers score: A comprehensive scoring system for early identification of sick mothers. J Obstet Anaesth Crit Care 2018;8:83-9
|How to cite this URL:|
Chakravarthy K, Pandya ST, Nirmalan PK. Development and validation of saving mothers score: A comprehensive scoring system for early identification of sick mothers. J Obstet Anaesth Crit Care [serial online] 2018 [cited 2019 Apr 24];8:83-9. Available from: http://www.joacc.com/text.asp?2018/8/2/83/242629
| Introduction|| |
Maternal mortality is a preventable cause of death in low- and middle-income economies. The enormous physiological drive and the associated physiological adaptations in pregnancy delay the recognition of early warning signs of sickness. Unrecognized deterioration of clinical status leads to worsening of illness in pregnant women. The 2003–2005 Confidential Enquiry into Maternal and Child Health (CEMACH) report recommended the use of a Modified Early Obstetric Warning System (MEOWS) in the acutely unwell parturient to facilitate early recognition of deterioration and further management.
There are certain limitations of MEOWS that lead to variations in its implementation. MEOWS is not globally popularized, it is basically a 'track and trigger' system for nurses to monitor sick women with established risk factors admitted in the ward, the associated trigger criteria may not be clear, the colour coding is not supplemented with a score, there is no clear indication of improvement or deterioration of the condition and MEOWS needs optimal training to reach 100% compliance.
A scoring system with universal application to detect and weigh the severity of sickness in pregnant women has not been developed in India. A comprehensive scoring system can segregate the pregnant women into low, moderate and high risk, recognise the sick mothers, assess the severity of sickness and help in triage and timely referral to a higher centre. This can contribute to the reduction of maternal morbidity and mortality at the community level.
The current study is the first attempt to arrive at a comprehensive scoring system – the Saving Mothers Score (SMS), for early identification of sick mothers. This article describes the development and validation of the SMS score.
| Methodology|| |
The SMS was developed through a study done at a single tertiary care centre in the public sector. The study protocol was approved by the institutional ethics committee and registered with the clinical trials registry – India (CTRI). The study included pregnant women of all age groups belonging to the American Society of Anaesthesiologists (ASA) grade I to grade IV, and admitted in the hospital during the period of 15th February 2016 to 15th Feb 2018. The study excluded, outpatients, pregnancy <6 weeks and those admitted for medical termination of pregnancy (MTP). The study was done in two parts. The first part involved the development and validation of the SMS chart (Feb 2016–2017). The second part of the study evaluated the implementation of SMS chart on reducing the morbidity and mortality of the pregnant women in two groups – a group that received care as per existing hospital norms of care and a second group where SMS charts were implemented in addition to existing hospital norms of care (Feb 2017–2018). The present article describes the first part of the study – development and validation of SMS chart.
The SMS included a combination of physiological assessment scoring and therapeutic weighted scoring to create a composite score that increases with severity of illness. The SMS chart was developed in three phases that included phase 1: formative research and generation of parameters of evaluation, phase 2: comprehensive item pool generation and phase 3: content validity or construct validity.
A literature search was done to retrieve articles of interest for the period January 1990 up to December 2015, using the PubMed, MD Consult, Cochrane Library and EMBASE databases. The keywords used for the search included pregnancy risk factors, early warning score, Modified Early Warning Score (MEWS), MEOWS, physiological parameters in pregnancy and morbidity and mortality in pregnancy. Parameters affecting the health of a pregnant woman were evaluated from the literature search and the normal and abnormal range (cut-off points or triggers) of the parameters were identified. The majority of the parameters were identified from the National Confidential Enquiry into Patient Outcome and Death, various forms of Early Warning Systems (EWS), MEWS, National Early Warning Score (NEWS) and MEOWS.
The draft SMS chart was subjected to content validation by 14 experts of similar cadre from the Departments of Anaesthesia and Obstetrics & Gynaecology. Five assistant professors in the Department of Anaesthesia and nine assistant professors in the Department of Obstetrics and Gynaecology participated in face-to-face structured interviews. Parameters included were judged on the basis of relevance. Acceptance of the parameter was conditional on an 80% minimum level of agreement between the experts.
The final draft SMS chart had 33 items that were pooled into three parameters: pregnancy-related risk factors, physiological variables and biochemical tests. There was 100% agreement on inclusion of the parameters and the cut-off points. Pregnancy-related risk factors were taken from the antenatal card and modified to include the obstetric and medical history. The parameters were given a bicolour coding with a simple yes or no as the potential response. An orange colour denotes the presence of a risk factor and a green colour indicates no risk. Each parameter was given a score of 1. The presence of ≥4 oranges at any time during pregnancy indicates a high-risk pregnancy. Eight physiological parameters were identified [blood pressure, pain, temperature, pulse, respiratory rate, saturation (SpO2), urine output and neurological status of the patient]. A triple colour coding, weightage and score was assigned to each one of them. Of the eight physiological parameters, respiratory rate and conscious level were given a weightage of 2. Other parameters were given a weightage of 1. Eight biochemical parameters were identified; colour coded and scores assigned depending on the severity of derangement [Figure 1] and [Figure 2].
The diagnostic effectiveness of the SMS chart was evaluated in a clinical setting. The sample size for the main study was estimated as 700 based on the prevalence of hypertensive disorders of pregnancy (HDP) in the hospital records (prevalence 35%) among the admitted cases, a 1:1 ratio of cases and controls, 5% margin of error and 95% confidence intervals (CI). The Raosoft – software was used to estimate the sample size. A convenience sample size (n = 120, one-third of the main study sample size estimated for the group where SMS chart was implemented) was used for the validation study. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and likelihood ratios were evaluated in 120 pregnant women selected through a simple random strategy.
Pregnancy-related risk factors were noted from the history and medical records. Blood pressure was measured using non-invasive blood pressure monitor, axillary temperature by digital thermometer, pulse, saturation and respiratory rate by pulse oximeter, conscious level by Alert, responds to Voice or Pain, and Unresponsive (AVPU) grade, degree of pain other than labour pain (0 = no pain, 1 = slight pain on movement, 2 = intermittent pain at rest/moderate pain on movement) and hourly urine output (in catheterized women) were noted. Total urinary output was measured and noted in women who were not catheterized. The parameters were noted once at admission, and thereafter at 6, 12, 24, 36 and 48 h in all the women. Colour coding was done and a score assigned.
A healthy pregnant woman was colour coded as green and had a score of 0–3. Pregnant women needing further management and/or a moderate risk (moderate deviation from the normal physiological parameters) and/or high dependency units (HDU) care were colour coded orange and had a score of 3–5. A sick pregnant woman/high risk/needing intensive care units (ICU) care and management (severe deviation from the normal physiological parameters) were colour coded red and had a score of ≥6.
Standard definitions for obstetric morbidity, obstetric haemorrhage, hypertensive disorders in pregnancy and sepsis were used in the study. Cardiovascular diseases, pulmonary complications, neurological complications, liver disorders, metabolic complications and other medical disorders complicating pregnancy were noted. Anaesthetic complications of interest included profound spinal hypotension, complications due to spinal/epidural, difficult airway and aspiration after difficult or failed intubation.
A trigger was defined as ≥4 oranges in the 'pregnancy-related risk factors' or any single red or 2 oranges in either 'physiological parameters' or 'biochemical parameters' or both. Obstetric intervention was categorized as caesarean section or assisted vaginal delivery (forceps or instrumental delivery). Transfer to medical unit referred to women who developed more than one organ dysfunction and needed medical intensive care. The proportion of women who triggered and developed morbidity was compared to the proportion of women who triggered but did not develop morbidity. The SMS was further validated by comparing it with the existing MEOWS.
Statistical analysis was done using the Stata statistical software version 10 (College Station, TX, USA). The diagnostic effectiveness for both trigger and score were derived independently for pregnancy-related risk factors, physiological parameters and biochemical parameters. An overall trigger and score was determined for all the three parameters combined together and the diagnostic effectiveness estimated. Validation of SMS was done by comparing the SMS sensitivity and specificity with that of the existing MEOWS.
| Results|| |
The majority of women (n = 113, 94.16%) were aged 20–34 years with a mean age of 23.5 years. About 81.6% were illiterates (n = 98) and of low socio-economic group (n = 115, 95.8%) [Table 1]. Anaemia was the most common pregnancy-associated risk factor (n = 34, 28.3%), followed by hypertensive disorders of pregnancy (n = 27, 22.5%) [Table 2]. Majority of the women had spontaneous vaginal delivery (n = 71, 59.2%) and the caesarean delivery rate was 34.2% (n = 41). The total number of high-risk pregnancies was 37 (30.83%) and 32 (86.49%) of the 37 women needed urgent obstetric intervention. The common obstetric and neonatal outcomes in the study subjects are presented in [Table 3].
Forty-six women (38.33%; 95% CI: 29.96, 47.26) triggered an alarm and 41 (81.93%, 95% CI: 77.54, 95.91) of these 46 women developed morbidity. The diagnostic effectiveness of SMS was comparable to MEOWS [Table 4]. The positive likelihood ratio for overall score was 7.39 (95% CI: 3.27, 16.7) and negative likelihood ratio was 0.48 (95% CI: 0.33, 0.67). The positive likelihood ratio for overall trigger was 45 (95% CI: 6.34, 320) and negative likelihood ratio was 0.40 (95% CI: 0.28, 0.57). The positive likelihood ratio for pregnancy-related risk factors was 5.99 (95% CI: 18.104.22.168) and negative likelihood ratio was 0.36 (95% CI: 0.23, 0.57). The positive likelihood ratio and negative likelihood ratio for physiological parameters score was 4.82 (95% CI: 1.61, 14.4) and 0.80 (95% CI: 0.66, 0.96), respectively. The positive likelihood ratio and negative likelihood ratio for physiological parameters trigger was 46.3 (95% CI: 2.83, 758) and 0.70 (95% CI: 0.57, 0.84), respectively. The positive likelihood ratio and negative likelihood ratio for biochemical parameters score was 43.8 (95% CI: 2.65, 725) and 0.73 (95% CI: 0.61, 0.88), respectively. The positive likelihood ratio and negative likelihood ratio for biochemical parameters trigger was 16.1 (95% CI: 2.13, 122) and 0.79 (95% CI: 0.68, 0.93), respectively. The SMS chart was validated against the existing MEOWS in predicting the maternal morbidity. The sensitivity of MEOWS was 54.6%, specificity 97.8%, PPV 92.5% and NPV was 79.9% when MEOWS was applied for the same parameters as SMS.
|Table 4: Diagnostic effectiveness of Saving Mother Score and validation against modified early obstetric warning system|
Click here to view
| Discussion|| |
Several domains including direct obstetric causes, access and availability of care, utilization of healthcare services, illiteracy and lack of protocols applicable to all levels of care contribute to the majority of maternal mortality in India. A uniform comprehensive scoring system may help the healthcare providers in early identification of the sick mothers.
The SMS showed low sensitivity of the physiological parameters, which may be attributable to the increased physiological reserve in the pregnant women that mask the early signs of illness. The specificity and PPV for both physiological and biochemical triggers was high. The return of an abnormal result in a test that has high specificity indicates that the person may have the disease or the condition of interest. This emphasizes the need for inclusion of pregnancy-related risk factors and biochemical parameters in early identification of sick mothers. The trigger, colour coding and the scoring were complementary in identifying the sick mother. The accuracy of the SMS chart was further reflected by the high PPV and NPV although the predictive values can be area or institute specific, based on the profile of women attending the clinic. The positive and negative likelihood ratios help to interpret diagnostic test results in a clinical setting. Higher values of positive likelihood ratios indicate a higher probability that the patient has the condition of interest. A positive likelihood ratio ≥10 can be used to make clinical decisions as the probability of disease increases by approximately 45% in such situations. The upper range of the 95% CI of positive likelihood ratios of all parameters or components of the SMS were ≥10 indicating that a positive result can be interpreted as an increased likelihood of high risk in this population.
The NEWS and MEOWS are used in UK for the assessment of sick patients and women admitted in the hospital. NEWS has six physiological parameters, three trigger levels and a red colour coding to alert the clinician. However, NEWS is not recommended for pregnant women. MEOWS includes the monitoring of physiological parameters, neurological response and status of lochia in the parturient. Singh et al. prospectively reviewed 676 consecutive obstetric admissions and concluded that MEOWS was 89% sensitive, 79% specific, with a PPV 39% and a NPV 98%. In the study by Singh et al., MEOWS chart was found to be 86.4% sensitive, 85.2% specific with a PPV and NPV of 53.8% and 96.9%, respectively for prediction of obstetric morbidity. The higher sensitivity of MEOWS may be related to the use of MEOWS to monitor the sick mothers identified and admitted in the ward, whereas SMS was used as a triage to identify the sick mothers. MEOWS is proposed to be used discreetly to monitor women with established risk of morbidity and has variation in implementation. Kyriacos et al. developed a MEWS with seven parameters for developing countries applicable to their country. We found comparable results when we validated SMS by applying MEOWS to the same parameters.
The ability to use SMS universally at the time of admission as a part of triage to identify sick mothers and to determine the severity of sickness is an advantage. A high-risk mother is one who has a score of ≥4 in the pregnancy-related risk factors itself. An additional risk is considered if she triggers 1 red or 2 oranges in the physiological parameters. It is mandatory, at this stage, to get biochemical investigations done or send the patient to a centre where the investigations can be done. If the patient is in a tertiary care centre with biochemical parameters the score can help in deciding whether the patient has to be sent to HDU or ICU care.
The study has certain limitations. The study centre is a tertiary referral centre for the care of pregnant women and children; however, the lack of support for other medical and surgical specialties (cardiology, nephrology, vascular surgery, endocrinology, urology and so on) limited the admission of severely sick mothers to this centre. There may be a selection bias as the study population is primarily from a lower socioeconomic group. The study centre and the study population may not be representative of a general population of pregnant women in India. Multi-centric studies in diverse settings are needed to further establish the strength of SMS chart in early identification of sick mothers.
| Conclusions|| |
The SMS is the first attempt to give a comprehensive scoring system for India that can be used for universal screening and identification of a sick mother to triage them, thus implementing early intervention. The magnitude of the illness can be judged both by colour coding and by the numerical score.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
World Health Organization. World Health Statistics 2016: Monitoring health for the SDGs Sustainable Development Goals. World Health Organization; 2016. Available from: http://www.who.int/iris/handle/10665/206498
. [Last accessed on 2018 Apr 04].
Soma-Pillay P, Nelson-Piercy C, Tolppanen H, Mebazaa A. Physiological changes in pregnancy. Cardiovasc J Afr 2016;27:89-94.
Al-Foudri H, Kevelighan E, Catling S. CEMACH 2003-5 saving mothers' lives: Lessons for anaesthetists. Contin Educ Anaesth Crit Care Pain 2010;10:81-7.
Goldhill DR, White SA, Sumner A. Physiological values and procedures in the 24 h before ICU admission from the ward. Anaesthesia 1999;54:529-34.
Adam S, Odell M. An acute problem? A report of the national confidential enquiry into patient outcome and death. Nurs Crit Care 2005;10:225-7.
Mitchell IA, McKay H, Van Leuvan C, Berry R, McCutcheon C, Avard B, et al.
A prospective controlled trial of the effect of a multi-faceted intervention on early recognition and intervention in deteriorating hospital patients. Resuscitation 2010;81:658-66.
Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling N. The value of modified early warning score (MEWS) in surgical in-patients: A prospective observational study. Ann R Coll Surg Engl 2006;88:571-5.
Williams B, Alberti G, Ball C, Bell D, Binks R, Durham L. National Early Warning Score (NEWS): Standardising the Assessment of Acute-Illness Severity in the NHS. London: The Royal College of Physicians; 2012.
Isaacs RA, Wee MY, Bick DE, Beake S, Sheppard ZA, Thomas S, et al.
A national survey of obstetric early warning systems in the United Kingdom: Five years on. Anaesthesia 2014;69:687-92.
Gao H, McDonnell A, Harrison DA, Moore T, Adam S, Daly K, et al.
Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med 2007;33:667-79.
Waterstone M, Bewley S, Wolfe C. Incidence and predictors of severe obstetric morbidity: Case-control study. BMJ 2001;322:1089-93.
Mercier FJ, Van de Velde M. Major obstetric hemorrhage. Anesthesiol Clin 2008;26:53-66, vi.
Mammaro A, Carrara S, Cavaliere A, Ermito S, Dinatale A, Pappalardo EM, et al.
Hypertensive disorders of pregnancy. J Prenat Med 2009;3:1-5.
Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, et al
. Surviving sepsis Campaign: International guidelines for management of sepsis and septic shock: 2016. Intensive Care Med 2017;43:304-77.
Kinsella SM, Winton AL, Mushambi MC, Ramaswamy K, Swales H, Quinn AC, et al.
Failed tracheal intubation during obstetric general anaesthesia: A literature review. Int J Obstet Anesth 2015;24:356-74.
Montgomery AL, Ram U, Kumar R, Jha P; Million Death Study Collaborators. Maternal mortality in India: Causes and healthcare service use based on a nationally representative survey. PLoS One 2014;9:e83331.
Meah VL, Cockcroft JR, Backx K, Shave R, Stöhr EJ. Cardiac output and related haemodynamics during pregnancy: A series of meta-analyses. Heart 2016;102:518-26.
Lalkhen AG, McCluskey A. Clinical tests: Sensitivity and specificity. Contin Educ Anaesth Crit Care Pain 2008;8:221-3.
Parikh R, Parikh S, Arun E, Thomas R. Likelihood ratios: Clinical application in day-to-day practice. Indian J Ophthalmol 2009;57:217-21.
] [Full text]
Sbiti-Rohr D, Kutz A, Christ-Crain M, Thomann R, Zimmerli W, Hoess C, et al.
The National Early Warning Score (NEWS) for outcome prediction in emergency department patients with community-acquired pneumonia: Results from a 6-year prospective cohort study. BMJ Open 2016;6:e011021.
Singh S, McGlennan A, England A, Simons R. A validation study of the CEMACH recommended modified early obstetric warning system (MEOWS). Anaesthesia 2012;67:12-8.
Singh A, Guleria K, Vaid NB, Jain S. Evaluation of maternal early obstetric warning system (MEOWS chart) as a predictor of obstetric morbidity: A prospective observational study. Eur J Obstet Gynecol Reprod Biol 2016;207:11-7.
Mackintosh N, Watson K, Rance S, Sandall J. Value of a modified early obstetric warning system (MEOWS) in managing maternal complications in the peripartum period: An ethnographic study. BMJ Qual Saf 2014;23:26-34.
Kyriacos U, Jelsma J, James M, Jordan S. Monitoring vital signs: Development of a modified early warning scoring (MEWS) system for general wards in a developing country. PLoS One 2014;9:e87073.
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3], [Table 4]