From Optimization to Control: An Algorithmic Perspective

Speaker: Prof. Peyman Mohajerin Esfahani (TU Delft - Delft Center for Systems and Control)
DEIB - Nicola Schiavoni Seminar Room (Building 20)
March 15, 2024 | 2:30 pm
Contacts: Prof. Maria Prandini
Research Line: Control Systems
DEIB - Nicola Schiavoni Seminar Room (Building 20)
March 15, 2024 | 2:30 pm
Contacts: Prof. Maria Prandini
Research Line: Control Systems
Abstract
In this talk, we draw an explicit analogy across four problem classes in optimization and control with a unified solution characterization. This viewpoint allows for a systematic transformation of algorithms from one domain to the other. With this in mind, we exploit two linear structural constraints specific to control problems with finite state-action pairs to approximate the Hessian in a second-order-type algorithm from optimization. This leads to novel first-order control algorithms with the same computational complexity as (model-based) value iteration and (model-free) Q-learning while they interestingly exhibit an empirical convergence behavior similar to (model-based) policy iteration and (model-free) Zap Q-learning with very low sensitivity to the discount factor. If time permits, we also discuss how an interesting analogy between the convex conjugate operator and the Fourier transform can reduce the typical time complexity of the dynamic programming operation from O(XU) to O(X + U) where X and U denote the size of the discrete state and input spaces, respectively.
Short Bio
Peyman Mohajerin Esfahani is an associate professor at the Delft Center for Systems and Control. He joined TU Delft in October 2016 as an assistant professor. Prior to that, he held several research appointments at EPFL, ETH Zurich, and MIT between 2014 and 2016. He received the BSc and MSc degrees from Sharif University of Technology, Iran, and the PhD degree from ETH Zurich. He currently serves as an associate editor of Operations Research, Transactions on Automatic Control, and Open Journal of Mathematical Optimization. He was one of the three finalists for the Young Researcher Prize in Continuous Optimization awarded by the Mathematical Optimization Society in 2016, and a recipient of the 2016 George S. Axelby Outstanding Paper Award from the IEEE Control Systems Society. He received the ERC Starting Grant and the INFORMS Frederick W. Lanchester Prize in 2020. He is the recipient of the 2022 European Control Award.