Implicitly Regularized RL with Implicit Q-Values. (arXiv:2108.07041v2 [cs.LG] UPDATED)

The $Q$-function is a central quantity in many Reinforcement Learning (RL)
algorithms for which RL agents behave following a (soft)-greedy policy w.r.t.
to $Q$. It is a powerful tool that allows action selection without a model of
the environment and even without explicitly modeling the policy. Yet, this
scheme can only be used in discrete action tasks, with small numbers of
actions, as the softmax cannot be computed exactly otherwise. Especially the
usage of function approximation, to deal with continuous …

arxiv rl values