OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Towards Reliable and Sample Efficient Deep Reinforcement Learning for Aerodynamics : a Benchmark Application

Berger, Sandrine and Arroyo-Ramo, Andrea and Guillet, Valentin and Jardin, Thierry and Rachelson, Emmanuel and Bauerheim, Michaël Towards Reliable and Sample Efficient Deep Reinforcement Learning for Aerodynamics : a Benchmark Application. (2021) In: 74th Annual Meeting of the APS Division of Fluid Dynamics, 21 November 2021 - 23 November 2021 (Phoenix, United States).

Full text not available from this repository.

Abstract

Deep Reinforcement Learning (DRL) recently led to new control solutions for dynamic systems across various domains but its application to Computational Fluid Dynamics (CFD) remains a challenge. DRL algorithms usually require a large number of samples, whereas the high computing cost associated to CFD limits the amount of data which can be produced. Therefore, it is crucial to establish (i) which algorithms are both sample efficient and reliable (ii) how control laws generalize from simple to more sophisticated environments on the same problem. This study targets specifically continuous actions algorithms with a replay buffer: DDPG, TD3 and SAC. The algorithms are trained and evaluated on an aerodynamic benchmark which consists in controlling the trajectory of a 2D airfoil. Trainings performed with a low-order model show that optimal control can be achieved with all three algorithms. Besides, SAC is found to be effective and reliable whereas DDPG and TD3 are prone to instabilities and half of the trainings do not converge to a proper solution. Finally, solutions obtained with the low-order model and with CFD on the same problem are compared and transfer of control solutions between the two modeling methods is discussed.

Item Type:Conference or Workshop Item (Paper)
Audience (conference):International conference proceedings
Uncontrolled Keywords:
Institution:Université de Toulouse > Institut Supérieur de l'Aéronautique et de l'Espace - ISAE-SUPAERO (FRANCE)
Laboratory name:
Statistics:download
Deposited On:08 Oct 2021 09:18

Repository Staff Only: item control page