Artificial Intelligence in HCI Best Paper Award

Paper Abstract
Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of natural language processing tasks. Their adaptability through in-context learning and fine-tuning has enabled applications in diverse domains. However, concerns regarding fairness, bias, and ethical implications remain significant. LLMs have been found to exhibit biases against marginalized groups, raising critical questions about their deployment in real-world scenarios. Recent advancements have enabled LLMs’ application in tabular classification tasks, an essential component in high-stakes domains. However, the fairness implications of LLMs in tabular data classification remain underexplored. This paper investigates whether social biases influence LLMs in this context and assesses their susceptibility to such biases. We conduct a comparative analysis of Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, and GPT-4, evaluating their accuracy and fairness in zero-shot, few-shot, and fine-tuned settings. Our results reveal significant social biases in the zero-shot setting, reflecting inherent biases from pre-training data. Few-shot learning moderately mitigates bias but at the cost of reduced accuracy, while fine-tuning with balanced datasets offers a more effective trade-off between fairness and predictive performance. These findings underscore the importance of bias mitigation strategies in tabular data classification and emphasize the need for regulatory frameworks to ensure the ethical deployment of LLMs. By highlighting the trade-offs between accuracy and fairness, this study contributes to the ongoing discourse on responsible AI development and deployment.

The full paper is available through SpringerLink, provided that you have proper access rights.