EURURO Vol. 72 No. 3

E70 476

Letter to the Editor
Reply to Tuomas Mirtti and Tero Aittokallio’s
Letter to the Editor re: Fatemeh Seyednasrollah,
Mehrad Mahmoudian, Liisa Rautakorpi, et al. How
Reliable are Trial-based Prognostic Models in
Real-world Patients with Metastatic Castration-resistant
Prostate Cancer? Eur Urol[1_TD$DIFF]. [3_TD$DIFF]2017;71:838–40. Clinical
Utility of Trial-estimated Prognostic Models
The letter from Mirtti and Aittokallio raises important
issues regarding reproducible research and the practical
utility of models in clinical decision-making. [6_TD$DIFF]However, their
request to consult model developers for model application
seems unwarranted. Benchmark models introduced in
high-impact journals, such as the one by [7_TD$DIFF]Aittokallio [8_TD$DIFF]et al
should be usable by others. The DREAM challenge
organizers made invaluable efforts to guarantee reproduc-
ibility, which was also a main rule for challenge participa-
tion. Duty falls on the developers themselves to ensure
models can be easily and correctly applied by others.
Missing values are intrinsic to clinical data, and clinically
relevant models should be able to robustly deal with any
missing values. Conventions in the real world (RW) and
randomized clinical trials (RCTs) are not consistent and there
are no strict universal rules. For instance, in the DREAM
challenge, two important predictors in the Halabi reference
model lactate dehydrogenase and albumin, were entirely
missing in the VENICE and ASCENT2 trials, respectively.
Regarding imputation, we indeed first carried out
imputation using each team’s own approach (teams
1 and 2, model-based imputation; team 3, median
imputation); only persisting missing values were median-
imputed. Of note, Aittokallio and co-authors used median
imputation for the Halabi model in although the original
study used improved model-based imputation . The
assumption that all patients had bone metastases was
based on their high prevalence in metastatic castration-
resistant prostate cancer (mCRPC), supported both by
literature and DREAM challenge data; 89% of patients
had bone metastases (ENTHUSE 33 and ENTHUSE M1 100%).
This high prevalence also drove the decision by Aittokallio’s
team 1 to remove bone lesions from their model.
Finally, Mirtti and Aittokallio [9_TD$DIFF]describe our RW data as
‘‘limited’’ data for a heterogeneous patient subpopulation
not attractive for application of mCRPC prognostic models.
This is somewhat misrepresentative. First, our data
comprise a 12-yr population-level cohort of all docetaxel-
treated mCRPC patients at Turku University Central
Hospital (n = 289), the only center providing cancer care
for a population of 470 000 in Southwestern Finland.
Second, we collected a set of clinical variables comparable
to the carefully designed RCTs. Third, while future assess-
ment of the models using other RW data would be
interesting, our current data already involve more patients
than those used by Aittokallio et al for independent RCT-
based validation in (ENTHUSE M1 trial, n = 266). Fourth,
the utility of RCT-tailored models in everyday practice
requires testing on RW data. As RCT inclusion and exclusion
criteria are strict, RCTs never represent the whole picture of
RW patients. Moreover, individual RCTs can differ greatly.
For instance, despite the standardized inclusion criteria of
the clinical trials in the DREAM challenge, the ASCENT2 trial
differed from the other RCTs and was in fact omitted by
Aittokallio’s team 1 from their final model construction .
In conclusion, the DREAM challenge was a valuable data-
sharing effort to build powerful prognostic models for mCRPC.
Our study confirmed the general utility of these trial-tailored
models in RWpatients but also highlighted the importance
of supplementing RCTs with RW data. Regarding Aittokallio’s
team 1 model, fulfillment of the ‘‘practical utility’’ criterion
would benefit from a reduction in the very high number of
predictors (>3000) and avoidance of any use of validation
data at any stage of model development.
Conflicts of interest: The authors have nothing to disclose[10_TD$DIFF].
Acknowledgments: Seyednasrollah received a grant from the Doctoral
Program in Mathematics and Computer Sciences at the University of
Turku. Rautakorpi received a grant from the Cancer Society of Finland. Elo
received grants from the European Research Council, European Union’s
Horizon 2020 Research and Innovation Program, Academy of Finland,
Juvenile Diabetes Research Foundation, and Sigrid Juselius Foundation.
The authors thank Aidan McGlinchey for checking the English language.
References
[1] Guinney J, Wang T, Laajala TD, et al. Prediction of overall survival for
patients with metastatic castration-resistant prostate cancer: devel-
opment of a prognostic model through a crowdsourced challenge
with open clinical trial data. Lancet Oncol 2016;18:132–42
.
available at
journal homepage:
DOIs of original articles: .
0302-2838/# 2017 European Association of Urology. Published by Elsevier B.V. All rights reserved.