Universidade Federal do Ceará
Mestrado e Doutorado em Ciências da Computação

Título: Fairness Evaluation in Large Language Models: An Analysis of Intrinsic Metrics

Data: 11/02/2025
Horário: 16h
Local: Sala de Seminários – Bloco 952

Resumo:

Large Language Models (LLMs) play a crucial role in various artificial intelligence applications, yet their fairness remains a significant challenge. This study investigates the effectiveness of intrinsic metrics for evaluating fairness in LLMs, focusing on similarity-based and probability-based approaches. Similarity-based metrics analyze bias in word and sentence vector representations, while probability-based metrics assess discrepancies in the probability distributions of model predictions. Our analysis explores the ability of these metrics to identify unwanted biases in diverse datasets, highlighting their limitations and potential in the fairness context. Additionally, we discuss how these metrics can be combined to provide a more robust assessment of model impartiality. The findings offer valuable insights for improving fairness practices in the development and evaluation of LLMs, contributing to the creation of fairer and more transparent models.

Banca examinadora:

• Prof. Dr. Javam de Castro Machado (MDCC/UFC) – Orientador
• Prof. Dr. José Maria da Silva Monteiro Filho (MDCC/UFC)
• Prof. Dr. Carlos de Oliveira Caminha Neto (UFC)
• Prof. Dr. Iago Castro Chaves (LSBD)