Universidade Federal do Ceará
Mestrado e Doutorado em Ciências da Computação

Título: Deep Representation Learning and Amortized Inference for Gaussian Processes

Data: 23/08/2024
Horário: 09h00
Local:  Sala de Seminários – Bloco 952 e Google Meet (meet.google.com/dej-nwbg-tfi)

Resumo:

Deep Kernel Learning (DKL) enables using Gaussian Process (GP) models for complex data domains (e.g. images and graphs) by coupling standard kernels and deep neural networks. However, training DKL is challenging and often leads to overfitting. Most notably, DKL often learns ’nonlocal’ kernels — incurring spurious correlations. On the other hand, from a sparse GP (SGP) perspective, a variational amortization of the inducing locations was introduced, called input-dependent SGP (IDSGP). Both methods are based on both GPs and Neural Networks (NNs), but with different points of view — while DKL brings representation learning for the input features, the IDSGP brings representation learning for the variational distribution. Inspired by both DKL and IDSGP, following the SGP framework, we introduce a coupling of DKL’s learned feature space and the inducing variables with the goal of avoiding the dominance of the ELBO’s data fitting term. The proposed amortized variational DKL (AVDKL) shares the feature representation learned through neural networks to warp the inputs to feed the GP kernel and to generate the inducing parameters via amortization. Such coupling prevents the input projection to a totally correlated and less useful feature space. We can summarize two main contributions. Firstly, we provide a study on the application of both DKL and IDSGP on semi-supervised node classification on graphs, a task in which, although there are many GP methods, the state-of-the-art is mainly dominated by graph neural networks (GNNs). In this study, we show that infusing GPs with a DeepGCN can bring better results compared to a standalone DeepGCN. Secondly, we introduce the AVDKL and conduct an extensive set of experiments to validate the proposed model. Based on our results, the AVDKL is generally better than its counterparts for tabular data; for graph node classification, AVDKL with a DeepGCN achieves slightly better accuracy than a standalone DeepGCN and is more calibrated on the majority of the datasets; for image classification, AVDKL with a ResNet-18 presents better results than a standalone ResNet-18.

Banca examinadora:

• Prof. Dr. César Lincoln Cavalcante Mattos (MDCC/UFC – Orientador)
• Prof. Dr. Diego Parente Paiva Mesquita (FGV – Coorientador)
• Prof. Dr. João Paulo do Vale Madeiro (MDCC/UFC)
• Prof. Dr. Denis Deratani Mauá (USP)
• Prof. Dr. Rafael Izbicki (UFSCar)