Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness

arXiv:2401.15127v2 Announce Type: replace-cross
Abstract: Knowledge sharing about emerging threats is crucial in the rapidly advancing field of cybersecurity and forms the foundation of Cyber Threat Intelligence (CTI). In this context, Large Language Models are becoming increasingly significant in the field of cybersecurity, presenting a wide range of opportunities. This study surveys the performance of ChatGPT, GPT4all, Dolly, Stanford Alpaca, Alpaca-LoRA, Falcon, and Vicuna chatbots in binary classification and Named Entity Recognition (NER) tasks performed using Open Source INTelligence (OSINT). …

abstract arxiv chatbots context cs.cl cs.cr cs.lg cyber cybersecurity cyber threat evaluation forms foundation intelligence knowledge language language models large language large language models llm llm chatbots opportunities presenting study threat threat intelligence threats type