Original research: Predicting pressure injury risk in hospitalised patients using machine learning with electronic health records: a US multilevel cohort study (2024)

STRENGTHS AND LIMITATIONS OF THIS STUDY

• We mined a cohort of over 35 000 hospitalisations from individual electronic health records to analyse changes in pressure injury risk over time.

• Machine learning methods were used to a develop a logistic algorithm for predicting categorical shifts between low risk and high risk for pressure injury among hospitalised patients.

• This machine learning algorithm outperforms standard of care (ie, risk assessment with the Braden Scale) to improve the prediction of patient risk for pressure injury.

• This study provides an algorithm that can be applied to predict pressure injury risk in current patient populations, as well as a general methodology for developing in-house predictive algorithms for rare patient safety events.

• A limitation of this study is inherent in biases of the Braden Scale towards Caucasian skin tones that may extend to this predictive algorithm based on its calibration data, unless future studies can cross-validate this algorithm with other races and ethnicities.

Study design

We applied supervised machine learning methods to an observational cohort of hospitalised patients in the USA to identify predictors of pressure injury risk and develop an algorithm to predict future risk. After multiple machine learning approaches were considered, we ultimately chose to develop an algorithm using multilevel modelling methods with predictors determined by random forests, in accordance with methodological guidelines prescribed by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) methodological task force on machine learning.¹⁹ We also followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guideline for reporting on this study.²⁰ Multilevel modelling offered improved individual-specific prediction capabilities relative to other methods tested with the data at hand.²¹ This approach also offered transparency to clinicians who may elect to use predictive analytics combined with clinical practice guidelines, such as validated risk assessment tools (eg, the Braden Scale), in order to make more informed decisions about patient care.²²

Data source

After receiving institutional review board (IRB) approval, we applied our method to the clinical data warehouses at two privately held US academic medical centre databases to collect samples of deidentified EHR data across four contiguous hospitals. An observational cohort of hospitalised patient EHRs over 5 years was used to obtain a substantial sample of calibration data for analytics from 2014 to 2019.

We identified 21 388 unique patients aged 18 years or older admitted to inpatient care at these facilities, including critical care, general medicine and surgery with at least 2 risk assessments for pressure injuries. The data were extracted from the Epic-based EHR system in use at these facilities. Data were gathered longitudinally by patient encounter every 12 hours during hospitalisation. Age and Braden risk score data for each 12-hour clinical shift up to the 12th shift (ie, 6 days total), and discharge diagnosis codes were collected at the patient- encounter level.

Study population

EHRs were included in the study in accordance with CMS performance guidelines for hospital quality issued by the US Agency for Healthcare Research and Quality as Patient-Safety Indicator #3 (PSI03, V.2020), which defines inclusion and exclusion criteria for hospital-acquired pressure injury.²³ A pressure injury is caused by damage to the skin and underlying tissue as a result of pressure and shearing forces; pressure injuries are measured in stages from 1 to 4, including unstageable, to describe advancement from low-stage pressure injury denoted by bruising and erythema, to high-stage pressure injuries that include open ulcerations that reach bone and muscle fascia.²⁴ PSI03 inclusion criteria for hospital-acquired pressure injury cases are as follows: stage 3, 4 or unstageable pressure injury not present-on-admission (not as a primary or secondary diagnosis), International Classification if Diseases Ninth-Revision (ICD-9, 707.x) and Tenth-Revision (ICD-10, L89.xx), per 1000 discharges among surgical and medical patients ages 18 years and older; excludes stays less than 3 days and certain diagnosis or procedural codes that place them at predisposed risk to skin injury: certain skin conditions (MDC 9); pregnancy (MDC 14); spinal cord injury (hemiplegia, paraplegia or quadriplegia); spina bifida; pedicled graft or debridement and a transfer between different facilities.

We modified some specifications of the PSI03 criteria to develop a predictive model that was inclusive of some high-risk patient populations, as consistent with prior informatics research on this patient population.¹⁰ For instance, we included patients who were hospitalised for at least 5 days and obtained 2 or more risk scores during admission. We also included patients with spinal cord injury in the study cohort since providers could benefit from a greater understanding of the association between risk and pressure injury, based on previous findings linking these outcomes.¹⁰

Risk assessment measure: the Braden Scale

The dataset contained Braden risk scores recorded by healthcare providers during every 12-hour clinical shift. The Braden Scale is an instrument that ranks risk on a scale from 6 to 23, with lower scores corresponding to higher pressure injury risk.⁹ A score of 18 or less is considered the threshold for follow-up with routine prevention and recurring risk assessment according to international clinical practice guidelines.⁸

Model development: logistic regression

To develop an algorithm for predicting an individual’s risk of pressure injury development, we explored multiple supervised methods in machine learning. The Braden Scale provided dichotomous sets of risk, lending well to the development of a logistic regression model to formulate a decision rule on categorical shifts in risk from low to higher risk states. The logistic features of the machine learning model were meant to be prescriptive to clinicians by providing a decision rule of which patients required follow-up within 2–5 days window as necessary.

We explored fixed-effects logistic regression in comparison to multilevel logistic regression with random-effects (aka ‘mixed-effects’). Mixed effects adjust models for subject-specific parameters, termed empirical Bayes estimates, in order to estimate changes in outcomes between individuals.²¹

Log-likelihood ratios (LLRs) were used in order to interpret the degree of fit of separate models. LLRs are based on a function of the error or variability in a particular model.²¹ LLR tests can be conducted to estimate the degree of improved fit between models, therefore, we used χ² tests of the LLRs to differentiate predictive performance between fixed-effects and random-effects models.²¹ We also used LLR to classify the degree of enhanced predictive accuracy of models with additional data. By minimising the number of data points needed to provide an accurate estimate of future risk, we hoped to reduce the total cost of continuous risk assessment for non-critical patients.

Model development: feature selection

To develop the predictive algorithm, the model necessitated features that captured patient-specific measurements of risk for pressure injury, as well as weighted estimates (ie, regression coefficients) using machine learning. Given the large sample size, we first applied established normalisation and regularisation techniques to the data structure to protect this analysis for risk of overfitting, in addition to cross-validation.²² Features that would serve as predictors were obtained using a variety of supervised methods (ie, random forests, support vector machines, boosting, bagging, LASSO and Ridge regression).^25–27 After testing these methods, we determined that random forests provided relevant predictors at reduced mean-squared error compared with other approaches. Predictors for the algorithm were derived from random forests applied to randomly selected groups of 1000 observations at a time, and regressing 10 variables in each run. We simulated this process for 20 000 iterations and maintained predictors in the final model that remained statistically significant (p<0.05). In addition, we maintained the use of time and age—due to health system restrictions on deidentification, age was the only piece of obtainable demographic data. The covariate time was an expression of hospital length of- stay for each patient, broken down into 12-hour shifts, and age was a fixed effect exploring the impact of age on risk since the Braden Scale does not adjust for either factor. However, age has been explored in several studies to have a strong association with pressure injury risk.^{7 10 28} These data were used to formulate a multilevel logistic regression model as framed below for predicting future risk (equation 1).

We tested the mixed effects form of the model for a random-intercept and random-slope focused on time. The empirical Bayes estimate for the intercept (u_i0) also accounted for within-subject correlation resulting from clustering encounters within patients, as it is hypothesised that patients with previous encounters resulting in a hospital-acquired pressure injury are greater risk for skin injury in a future encounter.^{10 21}

For the multilevel approach, we tested for the presence of correlations between concurrent measurements within observations, such as the degree of collinearity between risk scores measured using the Braden Scale. Correlation between time point measures represents a legitimate concern because clinicians can be compelled to score a patient similarly to a previous score if it is available, increasing the degree of autocorrelation over time.²¹ Various covariance structures were tested for in this approach, including independent, unstructured, autocorrelated and exponential. We also adjusted the quadrature of the models to achieve convergence. The final approach was configured to four quadrature points with independent covariance.

The regression approaches tested assumed linearity in subject-specific risk trajectories, and we applied quadratic random effects to samples to discern whether non-linearity could be observed.²¹ The absence of non-linearity would rule out the need for more sophisticated (ie, deep learning) approaches to statistical prediction.

Model evaluation and cross-validation

We applied constrained optimisation to the LLRs of each model to identify the optimal number of scores needed to efficiently and accurately predict future risk. By identifying the time point with the greatest marginal benefit to prediction accuracy, LLR, we identified the number of risk scores necessary for collection in a hospital setting to minimise overuse of costly skilled labour (ie, hospital staff) and resources.

Given the model’s binary outcome measure and training on the dataset, we performed cross-validation to the analysis by organising test and training sets to evaluate the predictive validity of the model. Data were randomly separated at the encounter level, with 60% of the data used for training the model and 40% of the data used for the validation set. We then predicted the results of the regression model on the validation set.

We explored model accuracy using sensitivity and specificity to plot a receiver operating characteristics curve (ROC) and measure area under the ROC (AUROC). In this analysis, improved sensitivity represents an increase in the identification of patients at risk of a pressure injury, given that risk actually does exist (ie, true positive); specificity represents a reduction in the designation of at-risk patients who are not in fact considered at risk (ie, false positive). This balance in improved sensitivity and sensitivity leads to health system efficiency since clinicians would be providing follow-up care to patients who actually have a higher likelihood of pressure injury risk. The AUROC was compared with a limited time-adjusted and age-adjusted model, and with the performance of the Braden Scale alone (ie, standard care). The AUROC value would guide our understanding of specificity and sensitivity of the predictive algorithm.

Financial impact

Pressure injury prevention is a costly protocol to implement on a daily basis. A typical hospitalised patient requires a routine that includes nursing time and material resources that can add up to US$99.44 per patient per day on average. Thus, a large 500-bed hospital could spend up to US$50 000 per day and US$18 million per year on this standard practice. Based on the assumption that a predictive algorithm could provide prescriptive follow-up for a select group of about only 25% of a typical patient population, we used an existing economic model in order to calculate the budget impact that this algorithm could save hospitals.²⁹ We assumed that the budget impact was based on 100% compliance with an algorithm-driven protocol, adjusting for the sensitivity and specificity of the algorithm. These economic modelling assumptions are consistent with previous economic evaluations on the cost-effectiveness of pressure injury prevention.^{29 30}

Patient and public involvement

We engaged stakeholders in the design and execution of this research through the ‘Pressure Ulcer Research on Patient Outcomes and Stakeholder Engagement’ (PURPOSE) Advisory Council at University of Southern California, which consists of patient advocates and provider stakeholders. On completion of external peer-review of this study, results were shared with the PURPOSE Advisory Council.

Patient population

We mined a cohort of patients to develop a predictive algorithm of pressure injury risk from clinical data warehouses at four hospitals affiliated with US academic medical centres. This provided us with 94 745 patient EHRs collected over a 5-year period. We analysed 11 different combinations of sequential risk scores over time taken from a sample of 21 388 hospitalised patients who had at least 5 hospital-days of data. These patients experienced 35 001 separate hospitalisations and 169 709 transfers between units, meaning that over 63% of hospitalisations were observed in individuals with more than one admission (table 1). These EHRs also recorded 1549 individual pressure injury cases, some of which occurred to the same patient on different parts of the body or during separate admissions.

The median age of these individuals was 58 years (mean: 56.6; range: 18–108). While the cut-off for minimum length of a hospital stay was 5 days, the average hospital length of stay was 18.7 days. Patients had pressure injury risk scores collected every 12 hours using the Braden Scale, meaning that over a 5-day period there were 10 risk scores collected.

Characterisation and optimisation of the predictive algorithm

The predictive algorithm was developed using risk scores across multiple time increments and evaluating the improved fit and predictive validity of the model with the addition of data from more time points, as well as additional predictors obtained through machine learning feature selection methods. We rejected the fixed-effects model in favour of a multilevel regression model with mixed-effects based on reduced error according to a χ² test of the (LLRs; p<0.001). The multilevel model provided different estimates of coefficients in predicting the trajectory of risk scores over time, but these coefficients were preferred since the random-intercept offered subject-specific estimates of risk.

A list of 40 predictors returned from machine learning was then narrowed based on clinical meaningfulness (ie, a reasonable causal relationship to pressure injury outcomes) and fit to the model using a stepwise approach. The approach returned 20 statistically significant and clinically meaningful predictors in addition to time and age from 4 categories: (a) prescription drugs—beta blockers, electrolytes, phosphate replacement, zinc replacement, erythropoietin stimulating agents, thiazide/diuretics, analgesics/narcotics, vasopressors; (B) diagnoses—dyspnoea, hyperlipidaemia, bed confinement, spinal cord injury and chronic osteomyelitis; (c) lab orders—urinalysis, lipid panel, prealbumin, stool culture, quantitative culture and stain with anaerobes and vancomycin random assay and (d) medical orders—upper extremity venous duplex exam.

A best-fit model was determined to possess risk scores between 5 and 6 time points, whereby enough risk scores reduced error and improved fit. A plot of the ROC for both models, combined with resulting area under the ROC (AUROC) showed that 5 and 6 time points were approximately equal (AUROC=0.73). Ultimately, the marginal benefit of an additional 12-hour shift may not have outweighed the associated costs of skilled labour and other resources used for an additional risk score recording 12 hours after the fifth risk score.

Predictive algorithm characteristics

For five risk assessments at equally timed intervals, we performed the multilevel logistic regression on the Braden Scale alone, then with time and age adjustments (table 2), and finally with the full model including random-forest-generated predictive features (table 3). A test of the LLRs for these models favoured the model with predictors generated from machine learning.

A plot of the ROC characteristics also favoured the machine learning algorithm (figure 1). In particular, at a specificity of 0.5, the machine learning model exceeded a sensitivity of 0.75; whereas the time-adjusted and age-adjusted model only reached a sensitivity of 0.596 at equal specificity. Standard care with the Braden Scale alone was lowest performing of these three information sets.

The multilevel regression approach assumed linearity in subject-specific risk trajectories. We tested whether there were elements of non-linearity in the subject-specific outcomes by applying quadratic random-effects to samples to discern whether non-linearity could be observed. The absence of non-linearity ruled out the need for more sophisticated (ie, deep learning) approaches to statistical prediction (figure 2). The use of a generalised linear model with maximum likelihood estimates was sufficient to capture the majority of the variance in our sample and is more easily interpretable and generalisable for policy and clinical decision-makers.

Cross-validation

Data from the hospital EHR were divided into training and testing datasets in order to conduct fivefold cross-validation on the accuracy and generalisability of the predictive algorithm. The training datasets contained 60% of the encounters, and testing dataset contained 40% of the encounters from the original sample. The test model consistently predicted the observed outcomes in the training model between 76.96% and 77.34% of the time for each of the five runs.

Financial impact

The predictive algorithm offers improved economic efficiency, such that a hospital could recoup substantial savings on a weekly basis. Since a risk assessment can take anywhere from 5 to 15 min per patient, this could represent up to 250 labour-hours in a single 500-bed facility per day, and between 30 000 and 90 000 labour-hours per year. An average 500-bed hospital would spend up to US$99.44 per patient per day on follow-up preventive tasks. For a hospital at 100% vol capacity, this represents a weekly investment of about US$348 000 or US$18 million annually. By comparison, a predictive algorithm that limits follow-up after the fifth time point reduces costs by over 48%; reduced follow-up after the sixth time point still can save as much as 42%. The 6% gain in economic efficiency between the fifth and sixth time point could represent a savings of about US$975 000 annually without substantial losses in sensitivity or specificity.

Limitations

There are several limitations to this study. First, there are inherent considerations about the reliability of Braden scores using EHR data. This predictive algorithm should be used in tandem with the Braden Scale and clinical judgement to identify patients who need immediate intervention in the first several days before transitioning individual patients to follow-up triggered by the EHR.⁶ Second, the retrospective nature of accessing these clinical data makes it difficult to verify the accuracy of Braden scores and other clinical data with respect to what was the true condition of the patient being observed. Third, the generalisability of this model outside of the observed academic medical centres is unknown and needs to be externally validated. Fourth, the study uses codes entered into a patient EHR in real time. There is potential bias in the entry of latter codes based on earlier diagnoses, prescriptions and orders. Fifth, given known biases of the Braden Scale towards Caucasian skin tones, there may be inherent bias in this predictive algorithm based on its calibration data, unless future studies of this algorithm can control for race and ethnicity.

Sixth, many other predictive features may exist, both within the EHR and unobserved. Our data request was limited to the listed classifications of covariates to simplify a complex data management process by as much as possible. Additionally, unobserved factors in the dataset such as race/ethnicity and sex may have also improved the model, but our IRB protocol limited access to multiple identifying factors at once.

Seventh, the choice to use logistic mixed regression for predictive analytics is a simple, supervised approach that fits within the ISPOR guideline for machine learning in outcomes research.¹⁹ While it depends on features provided from more sophisticated selection approaches (eg, random forests), the prediction method itself is not considered a modern, advanced machine learning approach.³³ Combining coefficients derived from this study with unsupervised methods (eg, deep learning) could represent an evolution in future research; however, it does violate the transparency and interpretability criteria we set with clinical collaborators for this study.

Eighth, the NPIAP Standardised Pressure Injury Prevention Protocol Checklist recommends the use of validated tools to measure nutrition and mobility levels.³⁴ While such tools were not captured in the EHR used in this study, our study team and PURPOSE Advisory Council reviewed resulting predictors from the random forest that could be helpful proxies for nutrition (eg, lab orders for prealbumin and lipid panels) and mobility (eg, bed confinement and spinal cord injury). Future research could improve on this analysis by including validated tools for mobility and nutrition in predictive analytics.

Lastly, a number of unique findings in this predictive algorithm could be considered paradoxical. Stool culture orders were associated with reduced risk, perhaps because despite having risk of bacterial infection and diarrhoea, ordering this lab within 5 days of admission places patients on a correct path to reduced pressure injury risk. Prescribed electrolyte replacement was protective, whereas phosphate replacement increased risk, perhaps differentiating the underlying risk factors of patients needing one replacement compared with the other. Exceptionally high ORs for bed confinement and spinal cord injury, despite small sample sizes, are worth additional cross-validation with other data samples given the small sample sizes of these data. While there is potential that some of these sets of predictors are highly correlated, our test for autocorrelation or non-parametric fit of the underlying covariance structure of the model returned no alternative findings.

Conclusions

Predictive analytics is now commonplace in hospitals to identify patients who are high risk for costly, harmful events and take steps to mitigate the circ*mstances of their risk.³⁵ Pressure injuries are particularly important in this light, causing 60 000 deaths per year and costing US healthcare systems over US$25 billion annually.³⁶ While pressure injury prevention represents a cost-effective alternative to treating the outcome, it requires investment in clinical staff and material resources to follow clinical practice guidelines. A predictive algorithm such as this could limit direct follow-up for pressure injury prevention to those patients of greatest need after several days of data collection, thereby reducing the costs of prevention by 40%–50%. These savings for hospitals would create financial bandwidth to justify the transition to EHR-based warning systems that use predictive algorithms to trigger response for patients at high risk of iatrogenic injury. Furthermore, pressure injury prevention programmes offer substantial financial savings that can be recouped to invest in other patient safety programmes in the future.

Reviewer comments:

Author'smanuscript:

Twitter: @DrWmPadula

Contributors: WVP, DGA, PJP and SS contributed to this manuscript in accordance with ICMJE: WVP, DGA, PJP and SS were involved in the concept and design of the study, critical revision of the paper for important and intellectual content, supervision and administrative support of the study. WVP was involved in the acquisition of data, statistical analysis and interpretation, and drafting of the manuscript. WVP and DGA were involved in obtaining funding for the resulting study. WVP is responsible for the overall content as the guarantor of the work, and accepts full responsibility for the work and/or conduct of the study, had access to the data, and controlled the decision to publish.

Funding: This study was supported by a grant from the National Institutes of Health Office of Extramural Research (KL2 TR001854, PI: WVP).

Competing interests: WVP declares personal fees and equity holdings from Stage Analytics. SS declares equity holdings from Bayesian Health.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review: Not commissioned; externally peer reviewed.

Patient consent for publication

Not applicable.

Ethics approval

Ethics approval was obtained from University of Chicago Medicine protocol #IRB14-0869; University of Southern California protocol #UP-20-00140.

1. AHRQ National Scorecard on Hospital-Acquired Conditions . Updated baseline rates and preliminary results 2014–2017 secondary updated baseline rates and preliminary results 2014–2017. 2019. Available: https://www.ahrq.gov/hai/pfp/index.html

2. Reddy M, Gill SS, Kalkar SR, et al.. Treatment of pressure ulcers: a systematic review. JAMA2008;300:2647–62. 10.1001/jama.2008.778 [PubMed] [CrossRef] [Google Scholar]

3. Lyder CH, Ayello EA. Pressure ulcers: A patient safety issue. In: Hughes RG, ed. Patient Safety and Quality: An Evidence-based Handbook for Nurses. Rockville, MD: Agency for Healthcare Research and Quality, 2008. [PubMed] [Google Scholar]

4. Padula WV, Delarmente BA. The National cost of hospital-acquired pressure injuries in the United States. Int Wound J2019;16:634–40. 10.1111/iwj.13071 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

5. Nussbaum SR, Carter MJ, Fife CE, et al.. An economic evaluation of the impact, cost, and Medicare policy implications of chronic Nonhealing wounds. Value in Health2018;21:27–32. 10.1016/j.jval.2017.07.007 [PubMed] [CrossRef] [Google Scholar]

6. Qaseem A, Mir TP, Starkey M, et al.. Risk assessment and prevention of pressure ulcers: a clinical practice guideline from the American college of physicians. Ann Intern Med2015;162:359–69. 10.7326/M14-1567 [PubMed] [CrossRef] [Google Scholar]

7. Creditor MC. Hazards of hospitalization of the elderly. Ann Intern Med1993;118:219–23. 10.7326/0003-4819-118-3-199302010-00011 [PubMed] [CrossRef] [Google Scholar]

8. Haesler E, ed. European Pressure Ulcer Advisory Panel, National Pressure Ulcer Advisory Panel, Pan-Pacific Pressure Injury Alliance. Prevention and Treatment of Pressure Ulcers/Injuries: Quick Reference Guide. Boston, MA: EPUAP/NPIAP/PPPIA, 2019. [Google Scholar]

9. Braden BJ, Bergstrom N. Predictive validity of Braden scale for pressure sore risk in a nursing home population. Res Nurs Health1994;17:459–70. 10.1002/nur.4770170609 [PubMed] [CrossRef] [Google Scholar]

10. Padula WV, Gibbons RD, Pronovost PJ, et al.. Using clinical data to predict high-cost performance coding issues associated with pressure ulcers: a Multilevel cohort model. J Am Med Inform Assoc2017;24:e95–102. 10.1093/jamia/ocw118 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

11. Rajkomar A, Oren E, Chen K, et al.. Scalable and accurate deep learning with electronic health records. NPJ Digit Med2018;1:18. 10.1038/s41746-018-0029-1 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

12. Churpek MM, Yuen TC, Winslow C, et al.. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med2016;44:368–74. 10.1097/CCM.0000000000001571 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

13. Ghanem-Zoubi NO, Vardi M, Laor A, et al.. Assessment of disease-severity scoring systems for patients with sepsis in general internal medicine departments. Crit Care2011;15:R95. 10.1186/cc10102 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

14. Barriere SL, Lowry SF. An overview of mortality risk prediction in sepsis. Crit Care Med1995;23:376–93. 10.1097/00003246-199502000-00026 [PubMed] [CrossRef] [Google Scholar]

15. Jones AE, Trzeciak S, Kline JA. The sequential organ failure assessment score for predicting outcome in patients with severe sepsis and evidence of hypoperfusion at the time of emergency Department presentation. Crit Care Med2009;37:1649–54. 10.1097/CCM.0b013e31819def97 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

16. Vorwerk C, Loryman B, Coats TJ, et al.. Prediction of mortality in adult emergency Department patients with sepsis. Emerg Med J2009;26:254–8. 10.1136/emj.2007.053298 [PubMed] [CrossRef] [Google Scholar]

17. Henry KE, Hager DN, Pronovost PJ, et al.. A targeted real-time early warning score (Trewscore) for septic shock. Sci Transl Med2015;7:299ra122. 10.1126/scitranslmed.aab3719 [PubMed] [CrossRef] [Google Scholar]

18. Alderden J, Pepper GA, Wilson A, et al.. Predicting pressure injury in critical care patients: A machine-learning model. American Journal of critical care: an official publication. American Association of Critical-Care Nurses2018;27:461–8. 10.4037/ajcc2018525 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

19. Padula WV, Kreif N, Vanness DJ, et al.. Machine learning methods in health economics and outcomes research-the PALISADE checklist: a good practices report of an ISPOR task force. Value Health2022;25:1063–80. 10.1016/j.jval.2022.03.022 [PubMed] [CrossRef] [Google Scholar]

20. Collins GS, Reitsma JB, Altman DG, et al.. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ2015;350:g7594. 10.1136/bmj.g7594 [PubMed] [CrossRef] [Google Scholar]

21. Hedeker D, Gibbons RD. Longitudinal Data Analysis. Hoboken, NJ: Jon Wiley & Sons, Inc, 2006. [Google Scholar]

22. Padula WV, Blackshaw L, Brindle CT, et al.. An approach to acquiring, Normalizing, and managing EHR data from a clinical data repository for studying pressure ulcer outcomes. J Wound Ostomy Continence Nurs2016;43:39–45. 10.1097/WON.0000000000000185 [PubMed] [CrossRef] [Google Scholar]

23. Agency for Healthcare Research and Quality (AHRQ) . Patient safety indicator 03 (PSI 03) pressure ulcer rate. AHRQ quality indicators (AHRQ QI) ICD-10-CM/PCS specification 2021 March. 2020.

24. Edsberg LE, Black JM, Goldberg M, et al.. Revised national pressure ulcer advisory panel pressure injury staging system: revised pressure injury staging system. J Wound Ostomy Continence Nurs2016;43:585–97. 10.1097/WON.0000000000000281 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

25. Le NQK, Do DT, Nguyen T-T-D, et al.. A sequence-based prediction of Kruppel-like factors proteins using Xgboost and Optimized features. Gene2021;787. 10.1016/j.gene.2021.145643 [PubMed] [CrossRef] [Google Scholar]

26. James G, Witten D, Hastie T, et al.. An introduction to statistical learning. In: An Introduction to Statistical Learning. New York, NY: Springer, 2013. 10.1007/978-1-4614-7138-7 [CrossRef] [Google Scholar]

27. Nguyen HS, Ho DKN, Nguyen NN, et al.. Predicting EGFR Mutation status in non-small cell lung cancer using artificial intelligence: A systematic review and meta-analysis. Acad Radiol2024;31:660–83. 10.1016/j.acra.2023.03.040 [PubMed] [CrossRef] [Google Scholar]

28. Allman RM. Pressure ulcers in the elderly. N Engl J Med1989;320:850–3. 10.1056/NEJM198903303201307 [PubMed] [CrossRef] [Google Scholar]

29. Padula WV, Pronovost PJ, Makic MBF, et al.. Value of hospital resources for effective pressure injury prevention: a cost-effectiveness analysis. BMJ Qual Saf2019;28:132–41. 10.1136/bmjqs-2017-007505 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

30. Padula WV, Mishra MK, Makic MBF, et al.. Improving the quality of pressure ulcer care with prevention: a cost-effectiveness analysis. Med Care2011;49:385–92. 10.1097/MLR.0b013e31820292b3 [PubMed] [CrossRef] [Google Scholar]

31. Padula WV, Armstrong DG, Goldman DP. Complexity bias in the prevention of iatrogenic injury: why specific harms may inhibit performance. Mayo Clin Proc2022;97:221–4. 10.1016/j.mayocp.2021.10.001 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

32. Padula WV, Black JM, Davidson PM, et al.. Adverse effects of the Medicare PSI-90 hospital penalty system on revenue-neutral hospital-acquired conditions. J Patient Saf2020;16:e97–102. 10.1097/PTS.0000000000000517 [PubMed] [CrossRef] [Google Scholar]

33. Lim T-S, Loh W-Y, Shih Y-S. A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification Algorithms. Mach Learn2000;40:203–28. 10.1023/A:1007608224229 [CrossRef] [Google Scholar]

34. Pittman J, Black JM, de Jesus A, et al.. The standardized pressure injury prevention protocol checklist 2.0: content validation. J Adv NursNovember23, 2023. 10.1111/jan.15964 [PubMed] [CrossRef] [Google Scholar]

35. Patel V, Jamoom E, Hsiao CJ, et al.. Variation in electronic health record adoption and readiness for meaningful use: 2008-2011. J Gen Intern Med2013;28:957–64. 10.1007/s11606-012-2324-x [PMC free article] [PubMed] [CrossRef] [Google Scholar]

36. Padula WV, Pronovost PJ. Addressing the Multisectoral impact of pressure injuries in the USA, UK and abroad. BMJ Qual Saf2019;28:132–41. [PubMed] [Google Scholar]