Evaluation of the modified quick sequential organ failure assessment scoring system for triage and prognostic assessment in canine emergency and critically ill patients: a retrospective study

Abstract

Background

In veterinary medicine, the qSOFA model has been studied in patients with conditions such as peritonitis or pyometra, and among the modified qSOFA models, only the qSOFA-lactate model has been researched. Thus, this study aimed to evaluate the effectiveness of q-SOFA-C-reactive protein (CRP), qSOFA-lactate, and quick systemic inflammatory response syndrome (qSIRS) models for triaging emergency and critically ill patients. These models were juxtaposed with conventional systems (SIRS, qSOFA, and acute patient physiology and laboratory evaluation [APPLE] fast) to ascertain their efficacy in patient triage and prognostication. In this retrospective cohort study, data from 166 dogs admitted to the Department of Emergency and Critical care at Konkuk Veterinary Medical Teaching Hospital between February 2021 and May 2023 were analyzed. Scoring systems were computed based on initial admission physical examinations (respiratory rate, heart rate, temperature, mentation, and systolic blood pressure) and laboratory results (white blood cell and platelet count and albumin, glucose, lactate, and CRP levels). Because no prior veterinary studies on the qSOFA-CRP model were available, optimal cutoff values were established using receiver operating characteristic (ROC) curves and the Youden index. Conventional scoring systems were compared with the modified qSOFA within the survivor and non-survivor groups. The most effective system was determined through ROC curve analysis.

Results

For the qSOFA-CRP model, we identified an optimal cutoff value for CRP at > 1.55 mg/dL. All modified qSOFA scoring systems showed significant differences between survivors and non-survivors, in contrast to the conventional scoring systems. Notably, the qSOFA-CRP model demonstrated the highest area under the ROC curve value (0.761, 95% CI 0.68–0.83) and odds ratio (13.373, p < 0.001) when evaluating mortality at 28 days.

Conclusions

The qSOFA-CRP model, when employing a CRP threshold of 1.55 mg/dL, demonstrated promising potential as a novel criterion for triaging emergency and critically ill patients. However, further assessment is required in a larger population of patients at the precise early stage of sepsis.

Background

Sepsis is a severe medical condition that results from an imbalanced host response to infection and is characterized by organ dysfunction [1,2,3,4]. When rapid treatment of sepsis is not achieved, tissue hypoperfusion and organ failure progresses to septic shock. The accurate and objective diagnosis of sepsis, confirmation of the presence of pathogens in the patient's blood, and identification of clinical symptoms indicative of infection are crucial. Blood cultures can be used for this purpose; however, they yield positive results in only approximately 15–49% of sepsis cases and even lower accuracy in patients who received antibiotics before testing [5, 6]. Furthermore, blood culture process typically required an average of 36 h to provide the results [7]; this entire process, including antibiotic susceptibility testing, can lead to delays in administering antibiotics to emergency patients. Therefore, several clinical variables and tools are used for early sepsis screening, including the systemic inflammatory response syndrome (SIRS) criteria, vital signs, signs of infection, sequential organ failure assessment (SOFA) criteria, and quick SOFA (qSOFA) score [8,9,10,11].

Among these tools, qSOFA is a simple and readily applicable scoring system for patient assessment that is based on three criteria: respiratory rate, altered mentation, and systolic blood pressure. This allows for a prompt evaluation without the need for specialized equipment or techniques or additional laboratory test, thereby aiding in the differentiation of patients who require emergency intervention [12]. Additionally, in human medicine, studies have shown that the application of qSOFA in emergency department patients helps identify those in need of immediate treatment, not only in patients with suspected infections but also in the non-infectious group. [13, 14]

However, the qSOFA and SIRS were designed to accurately identify early signs of infection related to respiratory and circulatory abnormalities rather than organ damage associated with severe infections. Consequently, they have been criticized for their low sensitivity and specificity in diagnosing sepsis alone, as they do not account for specific variables, such as the underlying comorbidities of patients [15,16,17,18].

The conventional scoring system APPLEfast provides a more comprehensive and objective assessment of a patient's condition by evaluating factors such as mentation, glucose, albumin, and lactate levels, and platelet count [19]. However, because it relies on multiple laboratory test results, it takes more time to obtain results compared to a physical examination. Additionally, in cases where blood sampling is not possible due to the patient's condition, scoring can be challenging.

Therefore, in human medicine, ongoing research is focused on developing a rapid and appropriate scoring system for the triage of emergency and critical patients by incorporating additional criteria into the existing qSOFA scoring system. To enhance the predictive accuracy of the qSOFA, comparative studies have evaluated the qSOFA-lactate [20, 21], qSOFA-CRP [22, 23], and qSIRS models, which combine qSOFA with the SIRS criteria [16, 17, 24] for patient assessment.

Nevertheless, in veterinary medicine, research related to qSOFA has only evaluated the prognosis of patients with peritonitis or pyometra [25, 26]. For modified qSOFA, studies have only been conducted on the qSOFA-lactate model [27]. Therefore, this study aims to assess the applicability of qSOFA in both infectious and non-infectious groups in veterinary medicine. Additionally, the study seeks to identify the most suitable modified qSOFA scoring system for triaging emergency and critically ill patients, including those with sepsis and in need of early treatment, by comparing it with conventional qSOFA, SIRS, and APPLEfast scoring systems [19].

Methods

The aim, design and setting of the study

This retrospective cohort study utilized the electronic medical records of canine patients admitted to the department of Emergency and Critical Care at Konkuk Veterinary Medical Teaching Hospital from February 2021 to May 2023, in order to calculate of all scoring systems. The animal-related procedures in this study were subjected to review and approval by the Institutional Animal Care and Use Committee (IACUC) at Konkuk University, under approval number [KU23197].

Patients

The inclusion criteria were the availability of values for the initial physical examination (respiratory rate, heart rate, temperature, mentation, and systolic blood pressure measured using a doppler device) and laboratory findings (white blood cells and platelet counts and albumin, glucose, lactate, and CRP levels, with CRP levels measured using the IDEXX Catalyst CRP Test, IDEXX Laboratories, Inc., Westbrook, ME, USA) at the time of first hospital admission. Hospitalization was administered to patients requiring close monitoring and treatment for conditions such as respiratory distress, hypotension, anemia, and decreased mentation, as identified through examinations upon admission. Data regarding the duration of hospitalization were also included. Additionally, data on patient signalment, age, sex, body weight, and outcomes were collected.

To compare the patient triaging capabilities and prognostic evaluation abilities of various scoring system, patients were divided into two groups based on the clinical oucomes: survivors and non-survivors. Survivors were defined as animals discharged from the hospital and remained alive for ≥ 28 days post-discharge. Non-survivors were defined as animals that either died during hospitalization or were euthanized owing to poor prognosis. Cases of euthanasia due to factors unrelated to the animal’s medical condition, such as financial constraints, were excluded.

Additionally, this study aims to evaluate the applicability of these scoring systems in general emergency and critically ill patients without confirmed infections. Patients were classified into infectious and non-infectious groups based on antibiotic usage during the treatment period. In this study, antibiotics were administered only to patients with confirmed infections, as evidenced by direct bacterial observation on microscopic smears of blood, urine, pleural fluid, ascitic fluid, or wound exudate, or by positive culture results.

Scoring systems

The SIRS, qSOFA, qSIRS, and APPLEfast scores were calculated based on individual data at the first hospital admission. Calculations for SIRS, qSOFA, qSIRS, qSOFA-lactate, and APPLEfast were based on previous studies[11, 19, 24, 27, 28]. The SIRS score was evaluated on a scale of 0–4 using four criteria:1) respiratory rate > 40/min, 2) white blood cell count < 5.0 K/uL or > 19.0 K/uL, 3) heart rate > 150 bpm, and 4) temperature < 37.2 or > 39.4 oC [9, 16, 17, 28, 29]. The qSOFA score ranges 0–3 based on the following criteria:1) respiratory rate > 22/min, 2) altered mentation, and 3) systolic blood pressure < 100 mmHg [28]. For the qSIRS score, six criteria were established by adding the SIRS criteria to qSOFA based on the approach used in a previous study in humans. The cutoff values were applied based on the results of previous studies in veterinary medicine [24]:1) respiratory rate > 22/min, 2) altered mentation, 3) systolic blood pressure < 100 mmHg, 4) heart rate > 150 bpm, 5) temperature < 37.2 or > 39.4 oC, and 6) white blood cell count < 5.0 or > 19.0 K/uL. The qSOFA-lactate score was calculated based on previous study criteria [27], where one point was added to the existing qSOFA score if the lactate level was > 3 mmol/L, resulting in a total score ranging 0–4 points. The APPLEfast score was determined according to established criteria, considering five factors: glucose, albumin, and lactate levels, platelet count, and mentation score, resulting in a score of 0–50 [19]. The criteria for the five scoring systems are listed in Table 1.

Table 1 SIRS, qSOFA, qSOFA-lactate, qSIRS and APPLEfast criteria definition

We employed ROC curves and the Youden index to determine the optimal cutoff value for the qSOFA-CRP model, as no prior studies in veterinary medicine were available. Subsequently, we incorporated this criterion into the three existing qSOFA criteria, making a total score of 0–4 points.

Statistical analysis

Statistical analyses were conducted using SPSS software (Version 23). Data normality was assessed using the Shapiro–Wilk test. For normally distributed data, the Student’s t-test was used to compare the difference between the two groups. Non-normally distributed data were compared using the Mann–Whitney U test. Accordingly, Variables with a normal distribution were presented as mean (SD), whereas those with a non-normal distribution were reported as median.

In the qSOFA-CRP model, we identified the optimal cutoff value for CRP using ROC curves to determine the point at which the Youden Index (J) was maximized. The Youden Index is the sum of sensitivity and specificity minus one (J = Sensitivity + Specificity −1).

To determine the scores with the highest sensitivity and specificity for each scoring system, we conducted ROC curve analysis and calculated the Youden index. Using this threshold score, we conducted AUROC analyses for each scoring system in relation to mortality and infection status. For scoring systems with an AUROC value greater than 0.5, we performed additional logistic regression analyses to compare odds ratios. This allowed us to identify the scoring system with the most significant triage capabilities. Additionally, ROC curve analysis was conducted to evaluate whether this scoring system could predict patient prognosis regardless of infection status. All ROC curve analysis was conducted with a 95% confidence interval, and statistical significance was set at p < 0.05.

Results

Case identification

During the study period, 166 dogs were admitted to our hospital. In total, 5 dogs were excluded because the absence of lactate (n = 3) and CRP (n = 1) measurements at admission and one patient was euthanized owing to financial constraints (n = 1). Finally, 161 dogs were included in this study.

Among the 161 dogs, 95 survived for up to 28 days after discharge, 64 died, and 2 were euthanized owing to poor prognosis; thus, 95 survivors and 66 non-survivors were included in the final analysis.

Comparison of physical examination findings (respiratory rate, heart rate, temperature, mentation, and systolic blood pressure) and laboratory results (white blood cell and platelet counts and albumin, glucose, lactate, and CRP levels) between survivors and non-survivors are presented in Table 2.

Table 2 Patients’ characteristics and comparison of physical and laboratory findings between survivors and non-survivors, with mean ± SD or median (range) calculations

Determination of cutoff value for qSOFA-CRP model

Receiver operating characteristic (ROC) curve analysis was performed to determine the optimal cutoff value for CRP in the qSOFA-CRP model. The results showed that CRP levels significantly differed between survivors and non-survivors (p < 0.001, AUC = 0.747, CI = 0.673–0.821). Using the Youden’s index, a cutoff value of 1.55 mg/dL was identified, providing a sensitivity of 88% and specificity of 58% (Fig. 1). Thus, we defined CRP concentrations > 1.55 mg/dL as an additional criterion in the qSOFA-CRP model, for which one point was added to the score.

Fig. 1

Receiver operator characteristic curve of C-reactive protein. Analysis of the receiver-operating characteristics was conducted to identify the optimal cut-off value in the qSOFA-CRP model. The maximum Youden’s index was observed at CRP > 1.55 mg/dL, confirming a sensitivity of 88% and specificity of 58%.; ROC, receiver operating characteristic curve

Comparison between survivors and non-survivors for each scoring system

For the SIRS, no statistically significant difference was observed between survivors and non-survivors, with both groups having a median score of 1 (range:0–4; p = 0.460). However, significant differences were observed between the two groups for the scores of qSOFA, qSIRS, APPLEfast, qSOFA-lactate, and qSOFA-CRP models. Survivors had a median qSOFA score of 1 (range:0–3), whereas non-survivors had a median score of 2 (range:0–3; p < 0.001). For the qSIRS, survivors and non-survivors had median scores of 2 (range:0–5) and 3 (range:0–6; p < 0.001), respectively. In the APPLEfast model, survivors had a median score of 20 (range:0–35), whereas non-survivors had a score of 21.5 (range:9–38) (p = 0.018). In the qSOFA-lactate model, survivors had a median score of 1 (range:0–4), whereas non-survivors had a score of 2 (range:0–4) (p < 0.001). Finally, for the qSOFA-CRP model, survivors had a median score of 2 (range:0–4), and non-survivors had a score of 3 (range:0–4) (p < 0.001) (Table 3).

Table 3 Comparison of scoring systems between survivor and non-survivor groups

ROC curve analysis was conducted to compare the area under the ROC (AUROC) and to identify the threshold scores for each scoring system. (Fig. 2). Using this threshold score, our study found that the conventional scoring systems, qSOFA (AUROC 0.65, 95% CI 0.55–0.73) and APPLEfast (AUROC 0.61, 95% CI 0.52–0.70), as well as the modified qSOFA scoring systems, qSOFA-CRP (AUROC 0.76, 95% CI 0.68–0.83), qSOFA-lactate (AUROC 0.70, 95% CI 0.61–0.77), and qSIRS models (AUROC 0.67, 95% CI 0.58–0.75), all had AUROC values greater than 0.5. This indicates that they have a significant ability to predict patient mortality.

Fig. 2

Receiver operator characteristic curve of SIRS, qSOFA, qSIRS, APPLEfast, qSOFA-lactate, qSOFA-CRP model. The individual area under the receiver operating characteristic curve (AUROC) values are as follows: SIRS (AUROC 0.53, 95% CI 0.44–0.62), qSOFA (AUROC 0.65, 95% CI 0.55–0.73), APPLEfast (AUROC 0.61, 95% CI 0.52–0.70), qSIRS (AUROC 0.67, 95% CI 0.52–0.75),qSOFA-lactate (AUROC 0.70, 95% CI 0.61–0.77), and qSOFA-CRP (AUROC 0.76, 95% CI 0.68–0.83).; ROC, receiver operating characteristic curve; SIRS, systemic inflammatory response syndrome; qSOFA, quick sequential organ failure score; qSIRS, SIRS + qSOFA; APPLE_fast, acute patient physiologic and laboratory evaluation score; qSOFA-lactate, modified qSOFA with lactate criteria added; qSOFA-CRP, modified qSOFA with CRP criteria added; CRP, C-reactive protein

Based on the identified threshold scores, the sensitivity, specificity, positive and negative predictive values, and odds ratios for each scoring system were calculated and are presented in Table 4.

Table 4 Contingency tables showing the sensitivity, specificity, positive and negative predictive values and odds ratios for mortality prediction

Additionally, logistic regression analysis was conducted regarding patient mortality prediction, considering age, weight, and each threshold score of qSOFA, APPLEfast, qSOFA-CRP, qSOFA-Lactate, and qSIRS scores. Overall, a predictive accuracy of 70.8% was confirmed (Hosmer–Lemeshow test, p = 0.536), and patient mortality was found to be associated with age (p = 0.004), qSIRS (p = 0.050), and qSOFA-CRP (p < 0.001) scores, with only age and qSOFA-CRP model being significant at a probability level of less than 0.05. Specifically, it was observed that patients with a qSOFA-CRP score of 2 or higher had a 9.834 times higher likelihood of mortality compared to those without.

Comparison of qSOFA and qSOFA-CRP scores based on infection status

During the treatment period, dogs were classified into an infection group (84 dogs) and a non-infection group (77 dogs) based on antibiotic usage. To evaluate the utility of the scoring system based on infection status using ROC curve analysis, the prognosis was assessed for qSOFA and qSOFA-CRP scores ≥ 2 in both groups. In the infection group, AUROC was 0.65 (95% CI: 0.54–0.75) for qSOFA ≥ 2 and 0.69 (95% CI: 0.61–0.76) for qSOFA-CRP ≥ 2 (Fig. 3a). In the non-infection group, AUROC was 0.60 (95% CI: 0.49–0.71) for qSOFA ≥ 2 and 0.71 (95% CI: 0.62–0.80) for qSOFA-CRP ≥ 2 (Fig. 3b).

Fig. 3

Receiver Operating Characteristic Curve of qSOFA and qSOFA-CRP Models Based on Infection Status. a. Area under the receiver operating characteristic curve(AUROC) values for qSOFA and qSOFA-CRP scores ≥ 2 in the infection group: qSOFA (0.65, 95% CI 0.54–0.75) and qSOFA-CRP (0.69, 95% CI 0.61–0.76). b. AUROC values for qSOFA and qSOFA-CRP scores ≥ 2 in the non-infection group: qSOFA (0.60, 95% CI 0.49–0.71) and qSOFA-CRP (0.71, 95% CI 0.62–0.80).; ROC, receiver operating characteristic curve; qSOFA, quick sequential organ failure score; qSOFA-CRP, modified qSOFA with CRP criteria added; CRP, C-reactive protein

Discussion

To our knowledge, the present study is the first investigation in veterinary medicine aimed at improving the qSOFA scoring system by introducing new criteria, such as CRP, lactate, and SIRS. By comparing these modifications with the conventional qSOFA, SIRS, and APPLEfast scoring systems, we aimed to identify a more appropriate scoring system for triaging emergency and critically ill veterinary patients.

The qSOFA scoring system was originally designed for sepsis diagnosis; however, in human medicine studies, it has demonstrated some potential for predicting prognosis and assisting in patient triage, even in cases where infection is not confirmed [13, 14]. In this study, ROC curve analysis indicated that qSOFA alone may not be sufficient as an independent prognostic marker, with AUROC values of 0.65 (95% CI: 0.54–0.75) in the infection group and 0.60 (95% CI: 0.49–0.71) in the non-infection group. However, the qSOFA-CRP model demonstrated higher AUROC values, with 0.69 (95% CI: 0.61–0.76) in the infection group and 0.71 (95% CI: 0.62–0.80) in the non-infection group. These findings suggest that the qSOFA-CRP model could serve as a novel tool for initial triage across a broader range of patients, regardless of infection status; however, further validation in a larger population is required.

Inflammatory response in the body is initiated by pathogen-associated molecular patterns (PAMPs) derived from invading pathogens or damage-associated molecular patterns (DAMPs) released from injured cells [30]. Induced by these factors, CRP is an acute-phase protein that typically begins to rise 4–6 h after the trigger, reaching its peak 24–48 h later. Therefore, it is used as an early biomarker for systemic inflammation [31,32,33].

In human studies, combining CRP level evaluation with qSOFA can more effectively diagnose patients than qSOFA alone [22, 23]. However, in veterinary medicine, no research has been conducted on qSOFA-CRP models. In the present study, we established a CRP threshold > 1.55 mg/dL based on patient outcomes using ROC curves to incorporate CRP as a new criterion. In comparison to conventional scoring systems (SIRS, qSOFA, APPLEfast), the qSOFA-CRP approach demonstrated the highest sensitivity and largest AUROC. This confirmed its significance as a criterion for triaging critical emergency patients, revealing a higher OR than qSOFA alone. Thus, the modified qSOFA model enhances the ability to identify emergency patients while improving prognostic capabilities.

Previous studies has investigated lactate as an indicator of shock, hypoxemia, and hypoperfusion and focused on elevated lactate levels and the association between lactate normalization time and mortality [34]. In both humans and animals, the use of lactate as a new criterion in the modified qSOFA scoring systems has been explored [20, 21, 27]. In a study with animals conducted by Ortolani et al., qSOFA alone did not significantly distinguish between survivors and non-survivors (p = 0.200). However, when lactate > 3 mmol/L was applied as a new criterion, it yielded the highest AUROC value (0.62, 95% CI 0.53–0.70). This finding underscores the significance of incorporating lactate level as an additional criterion in the modified qSOFA scoring system.

In our study, based on previous research findings, a lactate level of > 3 mmol/L was used as the cutoff value [27]. Using the qSOFA-lactate model for patient triage and prognosis assessment, we observed a significantly higher AUROC value (0.695, 95% CI 0.61–0.77) than conventional scoring systems, making it the second-highest AUROC value following the qSOFA–CRP model.

Although some studies have suggested that lactate level is more significant than CRP level in evaluating the prognosis of patients with sepsis [35, 36], our results indicated that the qSOFA-CRP model is a more sensitive indicator for triaging emergency and critically ill patients. This is because lactate tends to have higher concentrations as the disease progresses to severe sepsis or septic shock than the initial stages. Unlike CRP, which increases in concentration during the early inflammatory response to sepsis [37,38,39], lactate levels increase as a consequence of hypoxia due to reduced blood pressure and tissue perfusion that accompanies sepsis [40]. Therefore, although hyperlactatemia may be more significant in assessing patient outcomes, our results suggest that the qSOFA-CRP model is a superior scoring system for distinguishing early sepsis and triaging emergencies in critically ill patients.

We also observed that the qSIRS score had a higher AUROC value than qSOFA alone. However, when comparing ORs, qSIRS exhibited a lower predictive ability for patient outcomes than the qSOFA-CRP and qSOFA-lactate models. The SIRS was composed of four criteria: respiratory rate, white blood cell count, heart rate, and temperature. These criteria are characteristic features of infections; hence, most of the infected patients met the SIRS criteria, leading to the diagnosis of sepsis even in patients with less severe infections. This characteristic of SIRS results in low specificity for sepsis diagnosis [9,10,11, 29]. In contrast, qSOFA assesses both mentation and systolic blood pressure, allowing for a better reflection of sepsis-induced life-threatening organ dysfunction than SIRS [41]. However, qSOFA also has low specificity. To address these limitations and complement each other, research in humans has explored the "qSIRS" scoring system, which combines SIRS and qSOFA [24].

In our study, qSIRS demonstrated a larger AUROC value and higher OR than qSOFA alone, indicating an improved predictability for patient outcomes. However, similar to the limitations associated with SIRS, it exhibited the lowest specificity among conventional scoring systems and the modified qSOFA model. Therefore, its utility could be lower than that of other modified qSOFA models (qSOFA-CRP and qSOFA-lactate models).

The APPLEfast scoring system demonstrated the highest specificity of 80%. Although the modified qSOFA model aimed to address the limitations of the original qSOFA by incorporating new criteria, it relied predominantly on physical examinations. Consequently, it is perceived to have lower specificity than APPLEfast, which primarily employs laboratory test results. Thus, although the modified qSOFA model may be valuable for the rapid initial triage of emergency and critically ill patients at the time of admission, an additional laboratory test remains essential for a more accurate assessment of a patient’s condition and prognosis.

This study is limited to a single-center with a relatively small patient cohort, which could impact the predictive accuracy of the indices and models. Further validation of these models is essential through prospective studies involving a larger and more diverse patient population. Furthermore, since the evaluation of the qSOFA-CRP model was conducted within the same population used to determine its cut-off value, there is a possibility that it exhibited a higher AUROC compared to other scoring systems. Therefore, additional validation in an independent external population is necessary to more definitively assess its clinical utility.

The scoring was conducted based on lactate and CRP levels at the time of initial hospital admission, representing a single snapshot. Owing to the characteristics of a referral hospital, a significant proportion of the patients had received initial treatment elsewhere before admission. Therefore, it was challenging to determine whether the time of hospital admission reflects the actual onset of the disease, improvement phase, or advanced stage. Consequently, further investigations are needed to assess the clinical advantages of the modified qSOFA model for identifying patients in the early stages of sepsis.

Conclusion

Modified qSOFA models, including qSOFA-CRP, qSOFA-lactate, and qSIRS, predicted patient mortality significantly better than the conventional systems (SIRS, qSOFA, and APPLEfast) alone. Notably, the qSOFA-CRP model (CRP > 1.55 mg/dL) was identified as superior among all scoring systems, suggesting its potential for use as a new criterion for the triage and classification of emergency and critical patients. For additional validation of this scoring system, further studies involving a substantial number of patients in the true early stages of sepsis are required.