The predictive power of SAPS-3 and SOFA scores and their relations with patient outcomes in the Surgical Intensive Care Unit

Abstract

Objective: Individual risk of surgical patients is more often underestimated and there is not an absolute criterion demonstrating which patient deserves intensive care. Since a nominative assessment of these patients to quantify the intensity of critical illness is not appropriate, prognostic scores are used to assess the mortality rate and prognosis for critical patients including surgical ones. This study aimed to test the calibration power of SAPS-3 score and SOFA score of surgical patients undergoing gastrointestinal surgery, and identify any relation with patient outcomes in the department of surgical ICU.

Material and Methods: This retrospective observational study was conducted during the period between August 2017 and December 2017. It was performed at a Gastroenterological Surgical ICU, a tertiary care hospital in Ankara, Türkiye. To calculate SAPS-3 and SOFA score, physiological data and laboratory analysis on the day of ICU admission were used. Records were reviewed from hospitalization to medical discharge or hospital mortality. Statistical analysis included Mann Whitney U-test and ROC-curves to predict 30-day mortality.

Results: A total of 233 patients admitted to the Gastroenterological Surgical ICU were included into the study and the main reason for ICU admission was surgical problems. Mortality rate was 2.6 % (6 patients). Average SAPS -3 score was 32.5 and SOFA score was 30.1. A significant correlation was observed with the SAPS-3 score, but not with the SOFA score statistically in mortality as a dependent factor. The discriminative power, assessed using the AUC and the probability of death estimation, was satisfactory with the SAPS-3 scores (AUC 0.754) while it was lower with the SOFA score (AUC 0.631)

Conclusion: We found that SAPS-3 score was significantly correlated not only with mortality rate, but also with LOS in the ICU. Nonetheless, SOFA score was not related to mortality, but related to LOS in the ICU. Prognostic score systems are used to estimate mortality but they may be used to identify LOS in the ICU and postoperative complications. It can be concluded that SAPS-3 and SOFA scores may be used to prognosticate postoperative ICU requirement.

Keywords: Simplified acute physiology score, sequential organ failure assessment, surgical intensive care unit, mortality, predictive

Introduction

Surgical operations are classified as high-risk procedures and a significant proportion of intensive care unit (ICU) is composed of postoperative patients. These do not shadow the fact that the individual risk of surgical patients is more often underestimated, and less than 15% of patients who undergo those procedures have been admitted to an intensive care unit (1,2). There is not an absolute criterion or study demonstrating which patient deserves intensive care while debate is still continuing. The Society of Critical Care Medicine has published a guideline for ICU admission intending a better use of limited resources, but it has not gained popularity especially in surgical patients (3). So, risk adjustment via mortality prediction methods for this group of patients is a must.

Since a nominative assessment of these patients to quantify the intensity of critical illness is not appropriate, prognostic scores are used to assess the mortality rate and prognosis for critical patients including the surgical ones (4). Even though contradictious data are present in the literature, prognostic scoring systems (PSS) are used widely to make judgements in an objective manner (5). Among the PSSs, the Simplified Acute Physiology Score (SAPS) was investigated thoroughly and revised due to the improvements in health care (6,7). The revised version of this score system - SAPS-3 - uses 20 variables to predict hospital mortality from admission data (recorded within ± 1 h) (8). The Sequential Organ Failure Assessment (SOFA) score has primarily been proposed to assess the severity of organ dysfunction in septic patients in a sequential manner (9). Six organ systems (respiratory, cardiovascular, renal, hepatic, central nervous, coagulation) are taken into account, and the function of each is scored from 0 (normal function) to 4 (most abnormal). Serial changes in SOFA score over time are useful in predicting outcome. Its usage for mortality prediction based upon the fact that multiorgan dysfunction is commonly seen in critical situations like sepsis, chronic or acute liver failure, cancer or cardiac surgery (5,9).

Calibration and the discrimination are the two objective measures that evaluate the performance of prognostic models. Calibration -the relation of the estimated mortality with the observed mortality - is claimed to be as important as the discrimination - differentiation between survivors and nonsurvivors- in the means of the ICU eligibility of the patients and comparison of care between ICUs (7,10). Nonetheless, the calibration and discrimination power of the PSS can vary in time with the improvement in healthcare and it can vary across centers (11). As we searched in the literature, SAPS-3 and SOFA scores have not been evaluated in terms of sensitivity and specificity to predict mortality in postoperative patient population in a comparison manner. This study aimed to test the calibration power of SAPS-3 score and SOFA scores of surgical patients undergoing gastrointestinal surgery, and identify any relation with the patient outcomes in the department of surgical ICU.

Material and Methods

This retrospective observational study was conducted during the period between August 2017 and December 2017. It was performed at a Gastroenterological Surgical ICU, a tertiary care hospital in Ankara, Türkiye. The patients included in the study were those admitted to the surgical ICU for any reason after surgery. Patients with incomplete records and length of stay of less than 24 hours were excluded from the study. 

An extra formal consent other than the patients had given prior to hospitalization was not required for the current study because it was a case-control medical record review. Since our study was in the category of non-interventional clinical research with its retrospective structure, no ethics committee approval was applied. 

Data Acquisition

All clinical variables of the patients were retrospectively collected from our institutional database. Patients were analyzed on the following variables: diagnosis on admission, surgery type, medical history of the patients, reason for ICU admission, laboratory findings, mortality, length of stay (LOS) in ICU and in hospital at all. 

Surgery types were classified by localization of the pathology and divided mainly into four groups; biliopancreatic system (BPS), upper gastrointestinal system (GIS), lower GIS, and liver pathology. Reasons for ICU admission were classified mainly in four groups; cardiac problems like arrhythmia, respiratory problems like medical history of Chronic Obstructive Pulmonary Disease, hemodynamic problems like hypotension, and surgical problems like bleeding (bleeding from surgical site or any suspicion of bleeding, and observational admissions were classified in this category). Mortality as a variable in this study was described as death from any cause occurring within 30 days after surgery.

To calculate the SAPS-3 and SOFA score, physiological data and laboratory analysis on the day of ICU admission were used (postoperative 0). Records were reviewed from hospitalization to medical discharge or hospital mortality. Data were imported into a spreadsheet (Microsoft Excel 2013, Microsoft Corporation) for the calculation of the scores and their derived probabilities of death using the published equations and coefficients. To forestall the variability in the data collection, all values were reviewed by the authors of the study.

Statistical Analysis

Statistical analysis was performed using SPSS version 20.0 for Windows (SPSS Inc.; Chicago, IL , USA). Data were analyzed, and the continuous variables were reported as mean ± standard deviation (SD), and nominal variables were reported as total number and percentages. 

Variables were first evaluated by One-Sample Kolmogorov-Smirnov test as a normality test to choose the type of statistical tests -parametric or non-parametric test-, and the results showed asymp. Sig. (2-tailed) levels ≤ 0.05, so we decided to use non-parametric tests. For statistical analysis, correlations between variables were evaluated for significance by using the Spearman’s rho test. Categorical variables were evaluated by the Mann-Whitney U test of contingency. In all analyses, a ‘p’ value less than 0.05 was considered statistically significant and comparisons were 2-tailed.

Hosmer-Lemeshow test was used to calculate the calibration of SAPS-3 and SOFA test which express the ability of the test to determine the probability of death in accordance with the observed mortality. Discrimination was assessed using receiver operating characteristic (ROC) curves. The ROC curves were established as discrimination measurements with distributions per 10%, according to the predicted mortality and the obtained curve was appraised using the calculated area under the curve (AUC). AUC values > 0.75 was appraised as satisfactory, AUC values > 0.8 was appraised as well, and AUC values > 0.9 was appraised as very good.

Results

A total of 233 patients admitted to the Gastroenterological Surgical ICU were included in the study between August 2017 and December 2017. Patients older than 18 years of age, who stayed 24 h or more in the ICU, were included. Forty-nine patients’ records were excluded due to incomplete or unavailable data. Eight patients were younger than 18 years of age and excluded from the study. Patients’ demographic data and operational details of the groups (survivors versus non-survivors) are presented in Table 1.

Table 1. Demographic data and operational details of the groups

In Table 2, patients’ postoperative outcomes with prognostic scores related with mortality are evaluated. The average length of stay (LOS) in hospital was 14.35 days and LOS in ICU was 5.46 days. The average SAPS-3 score was 32.5 and SOFA score was 30.1. A significant correlation was observed with the SAPS-3 score, but not with the SOFA score statistically in mortality as a dependent factor.

Table 2. Postoperative outcomes with prognostic scores related with mortality*

Table 3. Postoperative outcomes with prognostic scores and relation with LOS in the ICU*

 The discriminative power assessed using the AUC and the probability of death estimation was satisfactory with SAPS-3 scores (AUC 0.754) while it was lower with SOFA score (AUC 0.631) (Figure 1 and Table 4). The Hosmer-Lemeshow goodness-of-fit test also revealed this with the discordance of these two scores with a good calibration for the SAPS-3 global model as shown in Table 4 with sig.level 0.757 versus 0.613.

Figure 1. ROC curve for SAPS-3 and SOFA scores.

Table 4. Hosmer and Lemeshow Test and Area under curve for SAPS 3 and SOFA score

Discussion

Due to the fact that surgical patients have different physiological and functional characteristics than other patients, estimation of prognosis and optimizing postsurgical care with lower misuse of ICUs are hard and complicated tasks (4). Prognostic scoring systems (PSS) like SAPS-3 and SOFA are utilized for this purpose in critical care medicine by disease severity assessment, comparing ICU performance and optimizing resource allocation. These systems use particular medical data to prognosticate an outcome’s possibility of occurrence (10,12,13 ).

SAPS-3 and SOFA scoring systems are simple and require non-sophisticated data, unlike other prognostic score systems. SOFA is mainly used to evaluate the clinical course sequentially and maintain a semiquantitative estimation of mortality. Therefore, it can be concluded that SOFA has a disadvantage in failing to show actual clinic state comparing to SAPS-3 (4,13). SAPS-3 utilization is claimed to be valid for surgical patients with good discrimination and calibration power, but the SOFA score has not been evaluated in surgical patients in our country. We compare these two commonly used PSS in mortality rate and in LOS in ICU mainly.

Mortality rates in the ICU differ between centers and patient admission criteria plays a very important role in this issue. Knaus et al. have described the lowest mortality rate as 6.4% in 42 ICUs, while in our study the mortality rate was 2.6 (14). This is probably due to the fact that in our study population, the major reason for the ICU admission was basic and observational reasons- surgical reasons- and we consider mortality as death from any cause occurring within 30 days after surgery. 

In our study, age factor as a variable did not affect mortality rate, but it was linked to prolonged LOS in the ICU, which was  in line with the finding in the study of Leong et al.  (15). 

Originally, SOFA score is computed 1 day after ICU admission and every 2 days subsequently. The highest scores are claimed to be most predictive of mortality (16). In our study, SOFA and SAPS-3 scores were calculated using data derived in the ICU admission day 0 retrospectively and the average scores were significantly higher in the non-surviving group. 

Discriminative power of the SAPS-3 was satisfactory and calibration was appropriate. Moreover, this model showed the relation of SAPS- 3 scores with mortality and length of stay in the ICU. AUC with SOFA score was found as 0.63,1 whereas it was found as 0.754 with SAPS-3. It is parallel to other studies that have found AUC 0.748 versus 0.810 (SOFA score versus SAPS-3 score) (16,17). Besides that, according to our findings, in this study SAPS-3 and SOFA are closely related to LOS in  the ICU for more than 3 days.

In a recent study, it has been demonstrated that approximately one fourth of surgical patients in the ICU were late admission- not admitted promptly postoperatively- and developed postoperative complications. In their study, Silva et al. have claimed that clinical evaluation underestimates the risk and has low predictive value, whereas PSS usage may be helpful in ICU referral and admission as an objective criterion (4). In another study, Nassar et al. have verified the feasibility of SAPS-3 in surgical patients and recommended its use as a distinction tool in ICU eligibility (18). PSS could be customized to advance their performance in specific patient populations (13). Our findings suggest that SAPS-3 or SOFA could be used as an admission criterion after proper customization.

Some studies have compared the SOFA score with another PSS-Acute Physiology and Chronic Health Disease Classification System (APACHE) score- as a mortality prediction model to predict hospital mortality and concluded that the combination of both scores enhances prognostic value (19,20). In another study, initial SOFA score has been found comparable to APACHE 3 in the ICU mortality prediction (16). 

The present study has some importance with the evaluation of different prognostic score systems in predicting mortality and relation to length of stay in ICU but it also has some limitations. First of all, one can claim that for comparison of different PSS, describing the sample size before the study is mandatory. As our study was designed in a retrospective manner this condition could not be achieved. Relatively small sample size limits the power of the analysis of goodness-of-fit Hosmer and Lemeshow Test which is poor to assess. Another potential limitation is being a single-center study with a different patients’ case mix as compared to the original SAPS-3 hospital outcome cohort. Lastly, the data collector reliability could be criticized in this study. Even though this is an important topic, we are quite reliant that, in this study, bias related to inadequate data collection was limited, since collection was done by the ICU doctors.

Conclusion

We found that SAPS-3 score was significantly correlated not only with mortality rate, but also with LOS in ICU. Nonetheless SOFA score was not related to mortality, but related to LOS in the ICU. PSS are used to estimate mortality but they may be used to identify LOS in the ICU and postoperative complications. It can be concluded that SAPS-3 and SOFA scores may be used to prognosticate postoperative ICU requirement.

Cite this article as: Mungan İ, Bektaş Ş, Altınkaya Çavuş M, Sarı S, Turan S. The predictive power of SAPS 3 and SOFA scores and their relations with patient outcomes in the surgical intensive care unit. Turk J Surg 2019; 35 (2): 124-130

Since our study was in the category of non-interventional clinical research with its retrospective structure, no ethics committee approval was applied.

In extra formal consent other than the patients had given prior to hospitalization was not required for the current study since it was a case-control medical record review. No change in our current clinical practice and no randomization was performed.

Externally peer-reviewed.

Consept - İ.M., Ş.B.; Design - İ.M., M.Ç.; Supervision - S.T., Ş.B; Resource - İ.M., S.S; Materials - İ.M., M.Ç.; Data Collection and/or Processing - İ.M., M.Ç., Ş.B., S.S; Analysis and Interpretation - İ.M., S.T., S.S.; Literature Search - İ.M., Ş.B.; Writing Manuscript - İ.M.; Critical Reviews - İ.M., S.T.

The authors have no conflicts of interest to declare.

The authors declared that this study has received no financial support.

References

1. Caldeira VM, Silva Junior JM, Oliveira AM, Rezende S, Araújo LA, Santana MR, et al. Criteria for patient admission to an intensive care unit and related mortality rates. Rev Assoc Med Bras 2010; 56: 528-34.
2. Jhanji S, Thomas B, Ely A, Watson D, Hinds CJ, Pearse RM. Mortality and utilisation of critical care resources amongst high-risk surgical patients in a large NHS trust. Anaesthesia 2008; 63: 695-700.
3. Nates JL, Nunnally M, Kleinpell R, Blosser S, Goldner J, Birriel B, et al. ICU admission, discharge, and triage guidelines: a framework to enhance clinical operations, development of institutional policies, and further research. Crit Care Med 2016; 44: 1553-602.
4. Silva JM, Rocha HMC, Katayama HT, Dias LF, de Paula MB, Andraus LMR, et al. SAPS 3 score as a predictive factor for postoperative referral to intensive care unit. Ann Intensive Care 2016; 6: 42.
5. Vincent JL, Moreno R. Clinical review: Scoring systems in the critically ill. Crit Care 2010; 14: 207.
6. Sakr Y, Krauss C, Amaral A, Réa-Neto A, Specht M, Reinhart K, et al. Comparison of the performance of SAPS II, SAPS 3, APACHE II, and their customized prognostic models in a surgical intensive care unit. Br J Anaesth 2008; 101: 798-803.
7. Poncet A, Perneger TV, Merlani P, Capuzzo M, Combescure C. Determinants of the calibration of SAPS II and SAPS 3 mortality scores in intensive care: a European multicenter study. Crit Care 2017; 21: 85.
8. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3--from evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med 2005; 31: 1345-55.
9. Jones AE, Trzeciak S, Kline JA. The Sequential Organ Failure Assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency department presentation. Crit Care Med 2009; 37: 1649-54.
10. Khan M, Maitree P, Radhika A. Evaluation and comparison of the three scoring systems at 24 and 48 h of admission for prediction of mortality in an Indian ICU: a prospective cohort study. Ain-Shams J Anaesthesiol 2015; 8: 294-300.
11. Haniffa R, Isaam I, De Silva AP, Dondorp AM, De Keizer NF. Performance of critical care prognostic scoring systems in low and middle-income countries: a systematic review. Crit Care 2018; 22: 18.
12. Rapsang AG, Shyam DC. Scoring systems in the intensive care unit: A compendium Indian J Crit Care Med 2014; 18: 220-8.
13. Strand K, Flaatten H. Severity scoring in the ICU: A review. Acta Anaesthesiol Scand 2008; 52: 467-78.
14. Knaus WA, Wagner DP, Zimmerman JE, Draper EA. Variations in mortality and length of stay in intensive care units. Ann Intern Med 1993; 118: 753-61.
15. Leong IY, Tai DY. Is increasing age associated with mortality in the critically ill elderly. Singapore Med J 2002; 43: 33-6.
16. Nair R, Bhandary NM, D’Souza AD. Initial Sequential Organ Failure Assessment score versus Simplified Acute Physiology score to analyze multiple organ dysfunction in infectious diseases in Intensive Care Unit. Indian J Crit Care Med 2016; 20: 210-5.
17. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996; 22: 707-10.
18. Nassar APJ, Malbouisson LM, Moreno R. Evaluation of simplified acute physiology score 3 performance: a systematic review of external validation studies. Crit Care 2014; 18: R117.
19. Minne L, Abu-Hanna A, de Jonge E. Evaluation of SOFA-based models for predicting mortality in the ICU: A systematic review. Crit Care 2008; 12: R161.
20. Sun D, Ding H, Zhao C, Li Y, Wang J, Yan J, et al. Value of SOFA, APACHE IV and SAPS II scoring systems in predicting short-term mortality in patients with acute myocarditis. Oncotarget 2017; 8: 63073-83.