Challenge data

Description

Competitive challenge

Biology

Health

Neuroscience

Regression

Tabular

Less than 10MB

Basic level

Dates

Started on Jan. 8, 2025

Challenge context

Parkinson's Disease (PD) is a common, complex, and disabling neurodegenerative disorder, placing an increasing personal, societal, and financial burden. Worldwide, more than 10 million people live with Parkinson’s disease, and this number is expected to rise due to aging populations. In France, approximately 200,000 people are affected, with an annual increase of around 25,000 new cases. Parkinson’s disease is the second most common neurodegenerative disorder globally, after Alzheimer’s disease. In 2005, the number of people over age 50 with Parkinson's disease (PD) in Western Europe’s five most populous nations and the world’s ten most populous nations ranged from 4.1 to 4.6 million. This number is projected to double to between 8.7 and 9.3 million by 2030.

The population of Parkinson's patients is highly heterogeneous, with multiple disease progression profiles. While symptomatic treatment is effective, no therapy currently exists to halt or slow the neurodegenerative process. Indeed, the disease begins decades before clinical symptoms are observed and would require an early profiling to hope for any preventive treatments. The underlying biology of the disease, phenotypic variation, and genetic risk profiles of patients must be explored to enable personalized therapeutic interventions.
One of the key characteristics of PD patients are the motor symptoms, encompassing rigidity, resting tremor, and bradykinesia (slowness of movement). These symptoms are related to the loss of dopaminergic neurons (neurodegeneration) in the brain. Dopamine is a neurotransmitter which participate in movement control by acting on the basal ganglia network (see figure 1).

![brain](/media/public/brain_20241122100308415845.png "brain")
 Figure 1- Anatomy of Parkinson's Disease - Key brain areas and mechanisms, From Summit For Stem Cell, https://www.summitforstemcell.org/glossary-of-parkinsons-disease/

The treatment is based on dopamine replacement therapies (DRT), which alleviate motor symptoms by restoring normal motor control. However, the response to the treatment changes during the progression of the disease, as illustrated in figure 2. While the treatment response is quite stable during the day at the early phase of the disease, the patients experiencee wearing off phenomenon (resurgence of symptoms at the end of the dose). At a more advanced stage, the patients fluctuate between a good motor state when the treatment is effective (“ON” state) and a bad motor state when the concentration of the medication is too low (“OFF” state). The “OFF” state thus represents the motor status related to the severity of the dopaminergic denervation (without treatment effect) and has been used as a surrogate of disease severity to monitor disease progression.

![level](/media/public/phase_20241122100325233822.png "level")
Figure 2 - Treatment response across different stages of Parkinson's disease

In clinical research, quantitative measurement of motor symptoms observed and quantified by the neurologist, can be assessed using an internationally validated standardised rating scale, the MDS-Unified ’arkinson's Disease Rating Scale (MDS-UPDRS). In addition to the rating of the motor severity observed by the neurologist, it also includes various aspects of Parkinson’s disease symptoms, such as motor and non-motor experiences of daily living and complications. More specifically, motor symptoms are assessed using a scoring system from 0 (normal) to 4 (severe) on 18 items (see figure 3 for an example), corresponding to the different motor aspects (akinesia, rigidity, tremor) and the different part of the body, leading to 33 subscores, for a total score between 0 and 132.
 
![q](/media/public/q_20241122095730078173.png "q")

Figure 3 - Example of assessment during motor examination for scoring Parkinson’s disease motor severity, From International Parkinson and Movement Disorder Society, https://www.movementdisorders.org/MDS/MDS-Rating-Scales/MDS-Unified-Parkinsons-Disease-Rating-Scale-MDS-UPDRS.html

Challenge goals

The evaluation of motor scores depends on several factors, including treatment response, patient age, disease duration, treatment dosage, and the time elapsed since the last medication intake. Typically, motor scores improve by 50% to 100% when transitioning from the OFF to the ON state.

In this challenge, we are particularly interested in the OFF motor score, as it can serve as a proxy for tracking the longitudinal progression of neurodegeneration and provide deeper insights into disease evolution.

However, several biases complicate the use of the OFF score, including human subjectivity during scoring, missing values due to incomplete or incorrect data, and variability caused by fluctuating levodopa blood levels. Furthermore, the OFF state is often undesirable for patients due to clinical discomfort and practical challenges, making it rarely assessed systematically. As a result, the ON score is usually the only score available.

The goal of this challenge is to predict and correct the OFF score for each patient visit. For the purposes of this challenge, a "true" OFF score has been estimated by removing biases, using real-world evidence and neurological expertise, as shown in figure 4. While this true OFF score is not available in practice and remains unknown to researchers, it has been approximated for this challenge. Participants are tasked with understanding this estimation process and developing models to automate the prediction of this new score.

![evo](/media/public/evol_20241122100401886361.png "evo")
Figure 4 - Example of PD patient motor score evolution over time, highlighting delays in drug intake and motor symptom variability
 
**Key Challenges: **
- Modeling Temporal Progression: Effectively capture and leverage the temporal evolution of motor scores to model the progression of motor symptoms over time for each patient.
- Adjusting for Drug Timing: Accurately account for the timing of drug intake by considering the delay between medication administration and clinical assessment to estimate the unbiased off motor score.
- Handling Missing Data: Develop robust methods to manage incomplete or irregularly collected data, ensuring accuracy and reliability despite data gaps.

Data description

The dataset provided for this challenge is synthetic, modeled after real-world data collected from multiple Parkinson’s disease cohorts. A cohort refers to a group of Parkinson’s disease patients who share common characteristics or conditions and are observed over time to study disease progression and treatment outcomes.

This synthetic dataset mirrors the structure, relationships, and missing data patterns found in actual patient records, offering participants a realistic yet anonymized dataset to work with. Each patient has multiple visits recorded at irregular intervals, and the dataset includes the following features for each visit, along with a unique patient ID and cohort ID for proper identification and tracking:

- Patient Demographics: Age, sex, and age at diagnosis (motor symptoms).
- Genetic Information: Relevant genetic markers.
- Drug Information: Dosage (in Levodopa Equivalent Daily Dosage).
- MDS-UPDRS Motor Scores: Both ON and OFF scores, with a recorded delay since the last drug intake.

In our modeling, the "wearing off" effect of the drug plays a crucial role, as it significantly influences both ON and OFF motor scores. Each patient is characterized by a unique time/efficacy response curve, derived from estimated pharmacokinetics and pharmacodynamics of levodopa in Parkinson’s disease. In this model, the drug’s effect is directly proportional to the concentration of levodopa in the blood, which changes as follow :After an initial absorption phase (quickly increasing concentration), the blood concentration of levodopa gradually declines toward zero. This decline enhances the accuracy of OFF score evaluations, as it provides a clearer representation of the patient’s baseline motor function without the drug's influence.

In this dataset, the response curve (pharmacodynamic) is assumed to remain constant over time, even as the patient ages (see the "honeymoon phase" in Figure 3). However, this is a simplification, as in real life, the response curve changes with disease progression and aging.

The generated dataset includes approximately 10,000 patients across multiple visits, resulting in nearly 100,000 entries. Each visit contains at least one motor score item (ON or OFF) and the target variable: the true/unbiased OFF score.

Benchmark description

To establish a baseline for this challenge, we propose a benchmark with a simple heuristic:  the baseline estimator will predict the “true” OFF score for a given patient as the mean OFF score, adjusted by the time since the onset of the disease plus the mean OFF score at diagnosis.

This benchmark provides participants with a starting point and a minimal performance threshold that can be easily surpassed. However, even though this model is simple, it is still a relevant tool for research, as it allows analysis of disease progression and offers participants initial insights into the type of function required to model PD evolution. Participants are expected to develop more sophisticated models that leverage the full set of features, handle missing data intelligently, and correct for biases in the timing of drug administration to make more accurate predictions.

---

### Evaluation Metric

This is a regression problem, and the primary evaluation metric will be the **Root Mean Square Error (RMSE)**. The RMSE will quantify the difference between the predicted “true” OFF scores and the actual values, providing a clear and simple measure of prediction accuracy.

The RMSE formula is defined as:

$$\text{RMSE}(y, \hat{y}) = \sqrt{\frac{\sum_{i=1}^{N} (y_i - \hat{y}_i)^2}{N}}$$

Where:  
- yᵢ: The target, i.e., the “true” OFF motor score.  
- ŷᵢ: The estimated score predicted by the participant’s model for a given visit *i* among the *N* available visits.

The objective for participants is to **minimize the RMSE**, ensuring that their predictions align closely with the true patient outcomes.

Files

Files are accessible when logged in and registered to the challenge

The challenge provider

Data Scientist at the Paris Brain Institute

PROVIDER WEBSITE

Challenge Data

Parkinson's Disease: Predicting and Correcting Bias in Motor Score Evaluation
by Institut du Cerveau

Description

Dates

Challenge context

Challenge goals

Data description

Benchmark description

Files

The challenge provider

Challenge Data

Parkinson's Disease: Predicting and Correcting Bias in Motor Score Evaluation by Institut du Cerveau

Description

Dates

Challenge context

Challenge goals

Data description

Benchmark description

Files

The challenge provider

Parkinson's Disease: Predicting and Correcting Bias in Motor Score Evaluation
by Institut du Cerveau