Efficient Model-Free Exploration in Low-Rank MDPs

arXiv:2307.03997v2 Announce Type: replace
Abstract: A major challenge in reinforcement learning is to develop practical, sample-efficient algorithms for exploration in high-dimensional domains where generalization and function approximation is required. Low-Rank Markov Decision Processes -- where transition probabilities admit a low-rank factorization based on an unknown feature embedding -- offer a simple, yet expressive framework for RL with function approximation, but existing algorithms are either (1) computationally intractable, or (2) reliant upon restrictive statistical assumptions such as latent variable structure, access …

abstract algorithms approximation arxiv challenge cs.lg decision domains embedding exploration factorization feature framework free function low major markov math.oc practical processes reinforcement reinforcement learning sample simple transition type