Reinforcement Learning for Solving Financial Problems

Abstract

This thesis explores the application of reinforcement learning (RL) to address two impor- tant financial problems: risk management and optimal trade execution. In risk management, we aim to balance returns with associated risks. To achieve this, we propose an enhanced RL model that integrates a dynamic Conditional Value at Risk (CVaR) measure. By leveraging distorted probability measures, CVaR allows the RL agent to emphasize worst-case scenarios, ensuring that potential losses are accounted for while optimizing long-term returns. Our method substantially reduces the model’s training time by efficiently reusing computation results, significantly lowering computational overhead. Furthermore, it optimizes the balance between exploration and exploitation. This approach leads to more robust decision-making in uncertain environments and a better overall return. For optimal trade execution, we formulate a flexible RL-based framework capable of dynamically adjusting to changing market conditions. Our model not only replicates the results of Almgren-Chriss model in linear environments but also demonstrates superior performance in more complex, nonlinear scenarios where traditional methods like Almgren- Chriss face challenges.