The Clinical Benefit of Fused AI Models: Improving PE Mortality Prediction Over Traditional Scores

Table of Links

Abstract

1. Introduction
2. Methods
3. Results
4. Discussion
5. Conclusions, Acknowledgments, and References

4. Discussion

4.1. Rationale for Study Approach

Prior studies have explored the potential of AI-based models in PE assessment, focusing on improving detection and diagnosis of PE.[11,25-27] Once a diagnosis of acute PE has been made, determining disease severity is important to guiding clinical management. PESI is a well-validated risk assessment tool for prediction of 30-day morbidity and mortality, commonly used in clinical practice.[5] In a meta-analysis including 71 studies and 44,298 patients, PESI and simplified PESI tools were the most highly-validated models available.[28] However, PESI’s positive predictive value for high-risk patients is only 11%.[6] Considering PESI’s limitations, we sought to develop AI-based models to build upon existing tools and improve prognostication.

While other studies have shown great potential for AI in detecting and diagnosing PE, few have shown benefits of AI for prognostication. Additionally, incorporation of multimodal data allows for more heterogeneous analysis. Somani et al. supported that use of fusion models may outperform non-fusion models in PE detection, supporting our efforts to assess efficacy of different permutations of fusion models in prognostication of PE.[25] A recent study explored the use of a multimodal model for PE risk stratification, based on prediction of thirtyday all-cause mortality.[29] A deep neural network (TabNet) was combined with a CNN, relying on a single binary label for each CTPA scan. Their fusion model achieved higher performance (AUC: 0.96) compared to clinical (0.87) and imaging (0.82) models. Our study further supports how multimodal approaches can improve healthcare decision-making and prognostication in PE patients.

4.2. Discussion of Study Findings

In this study, we showed that DL models incorporating combined imaging and clinical features can achieve high performance in predicting PE mortality, improving performance over PESI alone. The multimodal model outperformed both imaging and clinical models, indicating enhanced robustness from combining imaging and clinical data. The PESI-fused model slightly outperformed the multimodal model, indicating marginal benefit from incorporating the PESI framework. Models were also compared to RSF, with RSF outperforming the imaging model, clinical model, and PESI on the internal test set. However, RSF outperformed only the imaging model on the external test set. On both internal and external test sets, RSF was outperformed by the deep multimodal and PESI-fused models, demonstrating benefits of deep multimodal learning over a traditional survival method.

Given that PESI estimates the risk of 30-day mortality, additional survival comparison was conducted to evaluate 30-day performance. PESI demonstrated greater performance in predicting short-term PE survival compared to long-term, consistent with its clinical purpose. The clinical, multimodal, and PESI-fused models demonstrated improved performance in short-term prediction compared to long-term on the internal test set. However, they demonstrated lower performance compared to long-term on the external test set. Despite the improved performance of PESI in short-term prediction, the majority of DL models still demonstrated higher performance. On internal testing, clinical, multimodal, and PESI-fused models achieved higher c-indices than PESI. On external testing, PESI outperformed the clinical model but underperformed the multimodal and PESI-fused models. These findings indicate PESI’s performance is improved in short-term prediction. However, the deep multimodal and PESI-fused models still demonstrate improved performance, subject to model generalizability. This provides insight into how model performance may vary based on the specific outcome being assessed, as short-term mortality may be influenced less by competing risk factors.

For the clinical survival prediction model, we identified the predictive ability and importance of each feature. Age and history of cancer were found to have the greatest predictive ability. History of cancer had the greatest feature importance. This analysis may provide valuable insight into the underlying mechanisms or risk factors related to the predicted outcome. The alignment of our survival prediction model with observations in clinical practice provides further validation of model rationality.

NRI was used to analyze the contributions of different modalities to the multimodal framework, as well as the contribution of PESI to the PESI-fused model, by measuring the accuracy improvement achieved by incorporating each. The NRI values for +Clinical and +Imaging were positive, indicating improved performance from the incorporation of clinical and imaging data in the multimodal framework. Meanwhile, the values for +PESI were negative/lower, indicating less of a contribution. This suggests that the integration of imaging and clinical variables provides valuable and complementary information for survival prediction, resulting in more refined and reliable classification of individuals. Much of the information within PESI is already included in clinical variables, but conflicting characterization performance of PESI may lessen its contribution to the PESI-fused model.

Given the importance of RV dysfunction as a risk factor in PE patients, an additional factor-risk analysis was performed with the multimodal survival predictions. The multimodal survival model identified 68.8% of RV dysfunction patients as high-risk. The model also demonstrated a high correlation between high-risk identification and mortality, identifying 84.6% of mortality patients as high-risk. Through this risk stratification, the survival model was shown to be capable of predicting mortality, as well as having a relatively strong correlation with the prognostic factor of RV dysfunction. Thus, our model validates the association between RV dysfunction and death in PE patients.

4.3. Limitations

There are several limitations to this study. Like most DL-based survival analysis models, there is a concern for generalizability given that the model was trained using limited data from a single institution. To ensure external validity and generalizability of our models, we trained and validated them first on the single-institution internal dataset, then tested their accuracy on the previously-unseen multi-institution external dataset. Additionally, concatenation was used to fuse the two survival prediction branches- a more effective feature fusion mechanism between imaging and clinical data remains to be investigated. As we did not have access to data regarding patient treatment strategies within hospitals, we were not able to take clustering in treatment approaches into account. We were not able to compare the predictive value of the models between different settings (inpatient, emergency department, outpatient). Lastly, due to the CTPA requirement within our inclusion criteria, our study excludes more severe cases and perhaps the majority of PE mortality as these patients typically do not survive long enough to undergo CTPA.

Before being widely accepted, our models will likely require additional validation on larger and more diverse datasets, as well as prospective testing of the developed models. With the application of DL into medical care, appropriate and robust regulatory measures must be passed, and radiologists/clinicians will need to be trained to implement such models into their workflows.