Non-stationnary Lipschitz Bandits

In this presentation, we discuss the problem of non-stationary bandits with a continuous action space, where the reward function is Lipschitz-continuous and evolves over time. To build intuition and highlight the main sources of difficulty, we will begin by reviewing classical results on K-armed bandits, before moving on to Lipschitz bandits. In both cases, we present the classical Successive Elimination algorithm and discuss the optimal worst-case bounds. This analysis will provide insights into the appropriate notion of non-stationarity, how to detect it, and how to adapt effectively in a non-stationary environment.