Session III.4 - Foundations of Numerical PDEs
Monday, June 19, 17:30 ~ 18:00
Swarm-Based Random Descent Methods for Non-Convex Optimization
Eitan Tadmor
University of Maryland, USA - This email address is being protected from spambots. You need JavaScript enabled to view it.
We introduce a new swarm-based descent (SBD) method for non-convex optimization. The swarm consists of agents, each identified with position x and mass m. There are three key aspects to the SBD dynamics: (i) persistent transition of mass from high to lower ground; (ii) a random choice of descent marching direction, which is aligned with the orientation of the steepest gradient descent; and (iii) a time stepping protocol, h(x,m), which decreases with m. The interplay between positions and masses leads to dynamic distinction between `leaders’ and `explorers’. Heavier agents lead the swarm near local minima with small time steps. Lighter agents explore the landscape in random directions with large time steps, and lead to improve position, i.e., reduce the ‘loss’ for the swarm. Convergence analysis and numerical simulations demonstrate the effectiveness of SBD method as a global optimizer.