FY2022 Research Results(PDF 1.81MB).pdf

The Approximate Bayesian Inference Team, led by Mohammad Emtiyaz Khan, aims to develop AI systems that can learn and improve continually throughout their lives, like humans and animals, by using a new learning paradigm based on Bayesian principles. In 2022, the team made several research highlights, including:

Developing a robust deep-learning method related to Bayes and adding uncertainty to it
Extending the Bayesian learning rule using Lie-groups to simplify gradient computations and eliminate retractions
Designing a generally unimprovable procedure for relaxation time estimation in non-reversible Markov chains
Simplifying momentum-based Riemannian optimization over positive-semi-definite matrix submanifold
Using low-rank matrix completion techniques to reconstruct partially-observed high-dimensional time series

The team also made contributions to other areas, including:

Showing that Sharpness-Aware Minimization (SAM) is equivalent to an optimal relaxation of Bayes
Improving the accuracy of SAM by 8% over SGD, 10% over SAM-Adam, and 22% over Adam
Creating a dataset for African languages and analyzing features that contribute to cross-lingual transfer
Establishing deviation inequalities for stochastic approximation by averaging
Developing a method for high-dimensional time series completion using low-rank matrix completion techniques.