By submitting results to this competition, you consent to the public release of your scores on this website and at the SemEval-2024 workshop and in the associated proceedings, at the task organizers' discretion. Scores may include but are not limited to, automatic and manual quantitative judgments, qualitative judgments, and such other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers.
You further agree that the task organizers are under no obligation to release scores and that scores may be withheld if it is the task organizers' judgment that the submission was incomplete, erroneous, deceptive, or violated the letter or spirit of the competition's rules. The inclusion of a submission's scores is not an endorsement of a team or individual's submission, system, or science.
A participant can be involved in exactly one team (no more). If there are reasons why it makes sense for you to be on more than one team, then email us before the evaluation period begins. In special circumstances, this may be allowed.
Each team must create and use exactly one CodaLab account.
The team constitution (members of a team) cannot be changed after the evaluation period has begun.
During the development period:
Each team can submit as many as 999 submissions.
You will only be able to see the results of your submission and not others. Leaderboard is disabled
You will be able to see any warnings and errors for each of your submissions.
During the evaluation period:
Each team can submit only 3 submissions. However, only the final submission will be considered as the official submission to the competition.
You will be able to see any warnings and errors for each of your submissions.
Once the competition is over, we will release the gold labels and you will be able to determine the results of various system variants you may have developed. We encourage you to report results on all of your systems (or system variants) in the system-description paper. However, we will ask you to clearly indicate the result of your official submission.
We will make the final submissions of the teams public at some point after the evaluation period.
The organizers and their affiliated institutions make no warranties regarding the datasets provided, including but not limited to being correct or complete. They cannot be held liable for providing access to the datasets or the usage of the datasets.
Each task participant will be assigned another team’s system description papers for review. The papers will thus be peer-reviewed.
The dataset should only be used for scientific or research purposes. Any other use is explicitly prohibited.
The datasets must not be redistributed or shared in part or full with any third party. Redirect interested parties to this website.