Session II.4 - Foundations of Data Science and Machine Learning
Saturday, June 17, 15:00 ~ 15:30
Neural Networks as Sparse Local Lipschitz Functions: Robustness and Generalization
Jeremias Sulam
Johns Hopkins University, United States - This email address is being protected from spambots. You need JavaScript enabled to view it.
The theoretical underpinnings of deep neural networks continue to be poorly understood, despite their overwhelming role in top-performing algorithms in a plethora of applications. In this talk we will explore the benefits of studying deep relu networks from the perspective of sparse local Lipschitz functions. We will first show how the characterization of the local neighborhoods where these functions are locally Lipschitz, while preserving a certain degree of sparsity, allows us to develop tighter robustness certificates against input perturbations. We will then show how to use similar tools to provide tighter non-uniform, and non-vacuous, generalization bounds that scale better with width.
Joint work with Ramchandran Muthukumar (Jonhs Hopkins University).