TADPOLE challenge: Accurate alzheimer’s disease prediction through crowdsourced forecasting of future data

R. zvan V. Marinescu, Neil P. Oxtoby, Alexandra L. Young, Esther E. Bron, Arthur W. Toga, Michael W. Weiner, Frederik Barkhof, Nick C. Fox, Polina Golland, Stefan Klein, Daniel C. Alexander

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

The Alzheimer’s Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge compares the performance of algorithms at predicting the future evolution of individuals at risk of Alzheimer’s disease. TADPOLE Challenge participants train their models and algorithms on historical data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) study. Participants are then required to make forecasts of three key outcomes for ADNI-3 rollover participants: clinical diagnosis, Alzheimer’s Disease Assessment Scale Cognitive Subdomain (ADAS-Cog 13), and total volume of the ventricles – which are then compared with future measurements. Strong points of the challenge are that the test data did not exist at the time of forecasting (it was acquired afterwards), and that it focuses on the challenging problem of cohort selection for clinical trials by identifying fast progressors. The submission phase of TADPOLE was open until 15 November 2017; since then data has been acquired until April 2019 from 219 subjects with 223 clinical visits and 150 Magnetic Resonance Imaging (MRI) scans, which was used for the evaluation of the participants’ predictions. Thirty-three teams participated with a total of 92 submissions. No single submission was best at predicting all three outcomes. For diagnosis prediction, the best forecast (team Frog), which was based on gradient boosting, obtained a multiclass area under the receiver-operating curve (MAUC) of 0.931, while for ventricle prediction the best forecast (team EMC1), which was based on disease progression modelling and spline regression, obtained mean absolute error of 0.41% of total intracranial volume (ICV). For ADAS-Cog 13, no forecast was considerably better than the benchmark mixed effects model (BenchmarkME), provided to participants before the submission deadline. Further analysis can help understand which input features and algorithms are most suitable for Alzheimer’s disease prediction and for aiding patient stratification in clinical trials. The submission system remains open via the website: https://tadpole.grand-challenge.org/.
Original languageEnglish
Title of host publicationPredictive Intelligence in Medicine - 2nd International Workshop, PRIME 2019, Held in Conjunction with MICCAI 2019, Proceedings
EditorsIslem Rekik, Ehsan Adeli, Sang Hyun Park
PublisherSpringer
Pages1-10
Volume11843 LNCS
ISBN (Print)9783030322809
DOIs
Publication statusPublished - 2019
Event2nd International Workshop on Predictive Intelligence in Medicine, PRIME 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019 - Shenzhen, Switzerland
Duration: 13 Oct 201913 Oct 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference2nd International Workshop on Predictive Intelligence in Medicine, PRIME 2019, held in conjunction with the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019
CountrySwitzerland
CityShenzhen
Period13/10/201913/10/2019

Cite this

Marinescu, R. Z. V., Oxtoby, N. P., Young, A. L., Bron, E. E., Toga, A. W., Weiner, M. W., ... Alexander, D. C. (2019). TADPOLE challenge: Accurate alzheimer’s disease prediction through crowdsourced forecasting of future data. In I. Rekik, E. Adeli, & S. H. Park (Eds.), Predictive Intelligence in Medicine - 2nd International Workshop, PRIME 2019, Held in Conjunction with MICCAI 2019, Proceedings (Vol. 11843 LNCS, pp. 1-10). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Springer. https://doi.org/10.1007/978-3-030-32281-6_1