Switching the Loss Reduces the Cost in Batch Reinforcement Learning

arXiv:2403.05385v1 Announce Type: new
Abstract: We propose training fitted Q-iteration with log-loss (FQI-LOG) for batch reinforcement learning (RL). We show that the number of samples needed to learn a near-optimal policy with FQI-LOG scales with the accumulated cost of the optimal policy, which is zero in problems where acting optimally achieves the goal and incurs no cost. In doing so, we provide a general framework for proving $\textit{small-cost}$ bounds, i.e. bounds that scale with the optimal achievable cost, in batch …

abstract acting arxiv cost cs.lg iteration learn log-loss loss near policy reinforcement reinforcement learning samples show training type