HCI for Cybersecurity, Privacy and Trust Best Paper Award

Paper Abstract
Social engineering attacks such as phishing emails remain a critical method for cybercriminals to exploit sensitive data. Although the threat of AI-generated content in such attacks is growing, current training methods predominantly rely on simplistic human-designed emails. This research introduces a novel experimental paradigm to investigate differences in the detection of human-generated versus AI-generated phishing emails, as well as two different methods by which cyberattackers could use AI as a tool to generate phishing emails. Our behavioral results reveal that emails co-created by humans and Generative-AI models pose a greater challenge to end users compared to emails created by GPT-4 or Humans working alone. We also propose a cognitive model that predicts user behavior during training, which offers the potential to be used in future user training to improve training outcomes. Our work contributes by (1) identifying critical weaknesses in current social engineering training, (2) describing biases that human participants demonstrate when viewing GPT-4 written content in emails, and (3) proposing a cognitive model-driven solution to better train users against evolving threats.

The full paper is available through SpringerLink, provided that you have proper access rights.