SemEval-2024 Task 2: Safe Biomedical Natural Language Inference for Clinical Trials

arXiv:2404.04963v1 Announce Type: new
Abstract: Large Language Models (LLMs) are at the forefront of NLP achievements but fall short in dealing with shortcut learning, factual inconsistency, and vulnerability to adversarial inputs.These shortcomings are especially critical in medical contexts, where they can misrepresent actual model capabilities. Addressing this, we present SemEval-2024 Task 2: Safe Biomedical Natural Language Inference for ClinicalTrials. Our contributions include the refined NLI4CT-P dataset (i.e., Natural Language Inference for Clinical Trials - Perturbed), designed to challenge LLMs with …

abstract adversarial arxiv biomedical capabilities clinical clinical trials cs.ai cs.cl inference inputs language language models large language large language models llms medical natural natural language nlp safe shortcut type vulnerability