Presenter: Mirko Pieropan
Wednesday, July 29th, 2020 16:30
Location: Microsoft Teams – click here to join
Mirko Pieropan: Expectation Propagation for the diluted Bayesian classifier
Neural networks can learn a classification rule from examples by adapting their synaptic weights and are able to generalize to previously unseen data. We consider a diluted perceptron learning a classification rule from a teacher perceptron. The problem is formally equivalent to a framework known as one-bit compressed sensing. We propose an efficient computational scheme for sparse perceptron learning based on a message passing algorithm called Expectation Propagation (EP), assuming a Bernoulli-Gauss prior distribution for the weights to be learned by the student in order to enforce sparsity. We test the performance of the resulting algorithm when learning the classification rule from various kinds of patterns (i.i.d. Gaussian, correlated patterns from a multivariate normal distribution, temporal series generated by a recurrent network of diluted perceptrons) both in the absence and in the presence of noise affecting a given fraction of the labels. We compare the results with those obtained by training the student perceptron with other algorithms based on message passing (1bitAMP and grVAMP) and on expectation maximization (R1BCS). We also show that in many cases information about the density of the weights of the teacher perceptron and information on the noise affecting the labels are not required when training the student perceptron with EP, as they can be accurately estimated via maximum likelihood if enough patterns are provided and if the noise is not too large.
Biography: Mirko Pieropan received his M.Sc. degree in Physics of Complex Systems from Politecnico di Torino and a M2 degree in Physics from Université Paris VII – Denis Diderot (now called Université de Paris) in the context of a double degree program in 2016. He is now a final year Ph.D. student in Physics at Politecnico di Torino, working on linear estimation problems under the supervision of Andrea Pagnani and Alfredo Braunstein. He is a member of the SmartData@PoliTO center and co-organizes the SmartTalks series of webinars.