Digital Health Innovation and Informatics
SS 16 - Digital Health Innovation & Informatics 1
122 - Comparing the Kattan Nomogram to a Random Forest Model to Predict Post-Prostatectomy Pathology
Monday, October 22
5:15 PM - 5:25 PM
Location: Room 007 A/B
Comparing the Kattan Nomogram to a Random Forest Model to Predict Post-Prostatectomy Pathology
J. Kang1, C. W. Doucette1,2, I. El Naqa3, and H. Zhang1; 1Wilmot Cancer Institute, University of Rochester, Rochester, NY, 2University of Rochester Medical Center, Rochester, NY, 3Department of Radiation Oncology, University of Michigan, Ann Arbor, MI
In prostate cancer patients undergoing radical prostatectomy (RP), the Kattan pre-operative nomogram (KN) is an established tool for predicting organ-confined disease (OCD), extracapsular extension (ECE), positive lymph nodes (pN+) and seminal vesicle invasion (SVI). These predictions can help patients and providers guide treatment decisions. Here, we compare KN with a novel random forest (RF) model. We aim to improve on predictive performance by using a less-interpretable but potentially more predictive machine learning model in RF. We additionally derive variable importance using RF.
We retrospectively reviewed patients who underwent RP from June 2005 to May 2015 at our institution and included n=1560 patients with clinical intermediate risk prostate cancer (defined per NCCN guidelines). 60 predictors were collected that included demographics, PSA trends, and location-specific biopsy findings. MATLAB R2016b was used for analysis. Missing data was imputed. KN is a multivariable logistic regression model that uses 6 inputs: pre-biopsy PSA, primary and secondary Gleason, T-stage (T1a-T3c), and numbers of positive and negative cores. Beta coefficients for these variables were obtained from the official website (last updated May 31, 2016). RF is an ensemble decision tree method that uses bootstrapping and random variable splits at each node. The training set was 80% of patients. Parameters included 100 trees, 7 variables randomly split at each node, min leaf size 1, and maximum number of splits n-1. Out-of-bag estimates of feature importance were determined. Independent validation was performed on a holdout set of 20% of patients for RF. As KN is a pre-specified model, 100% of patients were used for validation. Receiver-operating-curve analysis using area-under-the-curve (AUC) was used to compare performance.
After surgery, 62.6% of patients had OCD. KN and RF predicted OCD with AUC 0.69 and 0.75, respectively. RF suggested the 3 most important variables are: % positive cores, % positive right base cores, and number of positive cores. ECE occurred in 36.9% of patients. KN and RF predicted this with AUC 0.69 and 0.73, respectively. RF model suggested the 3 most important variables are: % left mid positive cores, number of positive cores biopsied, and % positive cores. pN+ occurred in 3.6% of patients. KN and RF predicted this with AUC 0.71 and 0.64, respectively. SVI occurred in 6.9% of patients. KN and RF predicted this with AUC 0.72 and 0.69, respectively.
When positive and negatives outcomes are relatively balanced, as in OCD and ECE, then RF may be able to outperform KN by a small-moderate amount. This is likely aided by RF being trained on 60 variables, however, the data appears to be relatively noisy. In unbalanced outcomes that occurred in a relative minority of patients, like pN+ and SVI, KN may outperform RF. Future analysis will entail methods tailored for unbalanced, noisy data.
Author Disclosure: J. Kang: None. C.W. Doucette: None. I. El Naqa: None. H. Zhang: None.