Caltech Logo

Machine Learning and Scientific Computing Series

Tuesday, May 31, 2022
12:00pm to 1:00pm
Add to Cal
Online Event
Trustworthy Learning of Mechanical Systems, and Stiefel Optimization with Applications to Transformer and Optimal Transport
Molei Tao, Associate Professor, School of Mathematics, Georgia Institute of Technology,

The interaction of machine learning and dynamics can lead to both new methodology for dynamics, and deepened understanding and/or efficacious algorithms for machine learning. This talk will focus on such interactions centered around structured dynamics.
Specifically, I will first discuss data-driven learning and prediction of mechanical dynamics, for which I will demonstrate one strong benefit of having physics hard-wired into deep learning models; more precisely, how to make symplectic predictions, and how that provably improves the accuracy of long-time predictions.
Then I will report a construction of momentum-accelerated gradient descent algorithms on Riemannian manifolds, focusing on a particular case known as Stiefel manifold. The treatment will be based on, firstly, the design of continuous-time optimization dynamics on the manifold, and then a thoughtful time-discretization that preserves all geometric structures. Since Stiefel manifold corresponds to matrices that satisfy orthogonality constraint, two practical applications will also be described: (1) we markedly improved the performance of trained-from-scratch Vision Transformer by appropriately placing orthogonality into its self-attention mechanism, and (2) our optimizer also makes the useful notion of Projection Robust Wasserstein Distance for high-dim. optimal transport even more effective.

For more information, please contact Diana Bohler by phone at 626-395-1768 or by email at [email protected] or visit https://caltech.zoom.us/j/85710240262?pwd=NzlKVFUyY0RMMXVtcEp0eEh4dmtMZz09.