Imitation Learning in Discounted Linear MDPs without exploration assumptions

arXiv:2405.02181v1 Announce Type: new
Abstract: We present a new algorithm for imitation learning in infinite horizon linear MDPs dubbed ILARL which greatly improves the bound on the number of trajectories that the learner needs to sample from the environment. In particular, we remove exploration assumptions required in previous works and we improve the dependence on the desired accuracy $\epsilon$ from $\mathcal{O}\br{\epsilon^{-5}}$ to $\mathcal{O}\br{\epsilon^{-4}}$. Our result relies on a connection between imitation learning and online learning in MDPs with adversarial losses. …

abstract algorithm arxiv assumptions cs.lg environment exploration horizon imitation learning linear sample the environment type