Book · 2025
Prompt Recovery for Misinformation Detection at FIRE 2025
Proceedings of the 17th annual meeting of the Forum for Information Retrieval Evaluation
Abstract
This abstract provides a short overview of the first edition of the shared task on Prompt RecOvery for MisInformation Detection (PROMID) organized at the 17th Forum for Information Retrieval Evaluation (FIRE 2025). The shared task aims to deepen understanding how misinformation manifests in modern NLP systems. In this shared task we benchmark (i) prompt recovery, i.e. inferring the prompt that could have produced hallucinated content, (ii) factual incorrectness detection in machine-generated cross-lingual summaries and (iii) misinformation classification of tweets related to the 2022 Russo-Ukrainian conflict. A more detailed discussion is available in the track overview paper. For this shared task we received a total number of 16 submissions and 11 working notes. The best performing system for the factual correctness detection task in cross-lingual summaries employed a fine-tuned IndicBERTv2 model, while the best performing system for misinformation detection in tweets involved calibration of posterior probabilities obtained from a classifier head of a fine-tuned RoBERTa model.