SharKI

Shared Tasks as innovative approach for implementing AI- and Big Data-based applications in the higher education landscape

Project

The notion of "Shared Task" refers to a friendly scientific competition on a relevant research problem for which its participants develop computerized solutions. For example: "Given a set of arguments and a topic, classify these arguments into pro and contra regarding that topic." Submitted software solutions are evaluated by means of one or more measures of success. Afterwards the participants are invited to present their approach in a publication.

Shared tasks have a tradition in informatics, far beyond famous instances such as the World Computer Chess Championship (since 1974), the Loebner Prize Competition (since 1990) for the most human-like AI measured by a Turing test and the RoboCup in robotics (since 1996). Hundreds of shared tasks have been conducted in various fields of informatics, leading to many valuable, sometimes groundbreaking contributions.

Not only for that reason students are among those who participate with fascination and enthusiasm in such competitions. Hitherto, however, shared tasks play no role in didactics of informatics. Nevertheless, they have the potential to strengthen the union of research and teaching in accordance with Humboldt's Ideal.

In the SharKI project, we continue developing our unique in-house shared task platform TIRA further (see https://tira.io), aiming at enabling lecturers to setup, conduct and assess shared tasks as a novel teaching approach for their courses, thereby supported by AI- and Big Data-based features of the platform. Emphasis on a problem-oriented way of working justifies the classification of such courses into the teaching and learning model of problem-based learning. It further integrates methods such as exploring, experimenting, testing as well as game-based learning approaches. This promotes the development of learner skills such as flexibility, creativity and cooperation.

Publications

all publications
highlighted
- Theresa Elstner, Bärbel Hanle, Frank Loebe, Maik Fröbe, Nikolay Kolyada, Janis Mohr, Jörg Frochte, Sven Hofmann, Benno Stein, and Martin Potthast.
  Classification of Shared Tasks Used in Teaching. In Proceedings of the 2024 Innovation and Technology in Computer Science Education (ITiCSE 2024), 2024. ACM. [DOI]
- Maik Fröbe, Jan Heinrich Reimer, Sean MacAvaney, Niklas Deckers, Simon Reich, Janek Bevendorff, Benno Stein, Matthias Hagen, and Martin Potthast.
  The Information Retrieval Experiment Platform. In 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2023), pages 2826–2836, 2023. ACM. [DOI] → SIGIR 2023 Best Paper Award
- Maik Fröbe, Matti Wiegmann, Nikolay Kolyada, Bastian Grahm, Theresa Elstner, Frank Loebe, Matthias Hagen, Benno Stein, and Martin Potthast.
  Continuous Integration for Reproducible Shared Tasks with TIRA.io. In Advances in Information Retrieval. 45th European Conference on IR Research (ECIR 2023), Lecture Notes in Computer Science 13982, pages 236-241, 2023. Springer. [DOI]
- Theresa Elstner, Frank Loebe, Yamen Ajjour, Christopher Akiki, Alexander Bondarenko, Maik Fröbe, Lukas Gienapp, Nikolay Kolyada, Janis Mohr, Stephan Sandfuchs, Matti Wiegmann, Jörg Frochte, Nicola Ferro, Sven Hofmann, Benno Stein, Matthias Hagen, and Martin Potthast.
  Shared Tasks as Tutorials: A Methodical Approach. In Thirteenth AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 23), AAAI Conference on Artifical Intelligence, volume 37 no. 13, pages 15807-15815, 2023. AAAI Press. [PDF]

Material

project introduction [video - MP4, 11 MB, 2 min duration] [slides - PDF]
project overview at Leipzig University Teaching Day, 2022 [video - MP4, 20 MB, 6 min duration]
data protection declarations / informed consent
- Leipzig, April 2023 [in German]
- Leipzig, April 2022 [in German]
- Leipzig, October 2022 [in German, in English]
  (used in connection with the SemEval 2023 Clickbait Spoiling Challenge as shared task)

Events

02.-06.04.2023 – 45th European Conference on IR Research (ECIR 2023), Dublin, Ireland
11.-12.02.2023 – 13th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 23), Washington DC, USA [poster] [slides] [video - Youtube, 3 min duration]
15.-20.01.2023 – Dagstuhl Seminar 23031 "Frontiers of Information Access Experimentation for Research and Education", Schloss Dagstuhl, Wadern, Germany
22./23.06.2022 – KI-Vernetzungsveranstaltung, virtual
15.-17.06.2022 – Digital Cultures & Cultural Analytics: ARQUS Research Focus Forum on Digital Humanities, Leipzig, Germany
15.06.2022 – Tag der Lehre, Leipzig University, Germany
30.11.2021 – GI, Fachsymposium "Testen und Zertifizieren von KI-Systemen", virtual
04.11.2021 – University Future Festival, Workshop "Ethikkommission 2.0 – Diskussionen zum Einsatz von KI in der Hochschulbildung", virtual