Universidade Federal do Ceará
Mestrado e Doutorado em Ciências da Computação

Título: Deep Representation Learning and Amortized Inference for Gaussian Processes

Data: 10/07/2024
Horário: 14h
Local: Online (meet.google.com/aig-uqjs-nve)

Resumo:

Deep Kernel Learning (DKL) enables using Gaussian Process (GP) models for complex data domains (e.g. images and graphs) by coupling standard kernels and deep neural networks. However, training DKL is challenging and often leads to overfitting. Most notably, DKL often learns “non-local” kernels — incurring spurious correlations. On the other hand, from a sparse GP (SGP) perspective, a variational amortization of the inducing locations was introduced, called input-dependent SGP (IDSGP). Both methods are based on both GPs and Neural Networks (NNs), but with different points of views — while DKL brings representation learning for the input features, the IDSGP brings representation learning for the variational distribution. Inspired by both DKL and IDSGP, following the SGP framework, we introduce a coupling of DKL’s learned feature space and the inducing variables with the goal of avoiding the dominance of the ELBO’s data fitting term. The proposed amortized variational DKL (AVDKL) shares the feature representation learned via neural networks to warp both the inputs to feed the GP kernel and to generate the inducing parameters via amortization. Such coupling prevents the input projection to a totally correlated and less useful feature space. We can summarize two main contributions. Firstly we provide a study on the application of both DKL and IDSGP on semi-supervised node classification on graphs, a task in which, although there are many GP methods, the state-of-the-art is mainly dominated by graph neural networks (GNNs). In this study, we show that infusing GPs with a DeepGCN can bring better results when compared to a standalone DeepGCN. Secondly, we introduce the AVDKL and conduct an extensive set of experiments to validate the proposed model. In summary, AVDKL is in general better than its counterparts for tabular data; for graph node classification, AVDKL with a DeepGCN achieve slightly better accuracy than a standalone DeepGCN and is more calibrated on the majority of the datasets; for image classification, AVDKL with a ResNet-18 presents better results than a standalone ResNet-18.

Banca examinadora:

• Prof. Dr. César Lincoln Cavalcante Mattos (MDCC/UFC – Orientador)
• Prof. Dr. Diego Parente Paiva Mesquita (FGV – Coorientador)
• Prof. Dr. João Paulo do Vale Madeiro (UFC)
• Prof. Dr. Amauri Holanda de Souza Júnior (IFCE)