Frequently Asked Questions about Challenge Data.
The platform is open to everyone, students, professionals or beginners, professors and challenge providers. Registration is free for everyone.
Participant accounts are made for the actual competitors of the challenges, as well as students participating to a course created on the Challenge Data. Professor accounts allow professors to create course projects in which their students can enrol. They can track their students’ activity and rankings, and easily download their scientific reports. It is also possible to take part in challenges as simple competitors with these accounts. Challenge provider accounts are made for our partners who provide the datasets. It allows them to monitor the activity of their challenge through a private dashboard and to access live public and private rankings, as well as reports communicated by participants.
This means you may have already created an account. If you want to retrieve this account, try to reset your password (look at your spams). If you want to erase your former account, contact us through the contact form.
Go to the Challenges page. Select the challenge you want to participate in. If you participate as part of a specific course in an affiliated academic program, you must select the appropriate course in the list. Note that you cannot affect multiple challenges to a given course, so choose this challenge carefully.
Once you are registered to a challenge, you can submit a prediction file from the "The challenge" tab of a challenge. To know exactly what format your submission file should be, you can download the random submission file from the challenge page. Along with the prediction file, you can enter the name of the algorithm you used and the description of the parameters. It allows you to track your progress on this challenge. These will remain invisible to other participants, so do not hesitate to fill them precisely.
In order to avoid overfitting the test set, submissions are limited to 2 every 24 hours be it for a solo participant or a team.
You receive the score of your submission on the public part of the the test set.
No, we only store the score your submission made.
According the free exchange spirit of our initiative, participants to a challenge are invited to write a report as a scientific article, analyzing and explaining the performance of their algorithms, and providing references of published articles used, and upload it on the platform at the end of a challenge. This report can be made public or available only to challenge providers. Students registered to a course can on top submit a report at any time during the course which will be available to professors, and can elect to make it as well public or available to challenge providers, fostering the free exchange principle of this initiative.
In addition to the potential rewards offered by challenge providers on each challenge during private award ceremonies, the Challenge data organization will reward best participants (or team of participants) of each challenge at the end of a season during a ceremony at Collège de France as part of Professor Stéphane Mallat course. Details about this event shall be timely published on the Home page.
The test set is split into a public set and a private set. All your submissions are evaluated on both part of the test set. By default we take your best submission on the public set for rankings. The public leaderboards is the ranking of all participants’ submission on the public set. It is actualized after every submission. The private leaderboard is the rankings of participant’s submission on the private set, it is hidden from you and used for evaluation.
For every competitor, private leaderboards are published twice a year, on June the 15th and December the 15th at GMT+00. Pay attention to the fact that the leaderboard may not be published at midnight if your local time isn’t GMT+00.
Yes, you can still submit solutions and see the score you got on the public set of the data.
Participants can compete on past challenges the same way they would compete on current year challenges. The public and private leaderboard are continuously running. For all challenges, past and current, a private leaderboard is released to participants twice a year, on the 15th of June and 15th of December. For current challenges the December leaderboard is used to reward best participants during a ceremony at Collège de France.
Yes, you have access to challenges released this year and to the challenges from previous editions.
It is a date at which participants of a course will be able to see their score on the private set of data, other than the end date of your course. It enables students to know if their model has the same behaviour on the private set as on the public set or if it overfits the public data.
Most of the time it is a format error. Make sure your file has the right format, you can infer it by downloading the random submission file. Another common cause is that for some challenges the computation of your score may take some time. If it fails, try again. Otherwise you can send us an email using “Contact” tab.