Occupancy Information Ratio: Infinite-Horizon, Information-Directed, Parameterized Policy Search. (arXiv:2201.08832v1 [cs.LG])

We develop a new measure of the exploration/exploitation trade-off in
infinite-horizon reinforcement learning problems called the occupancy
information ratio (OIR), which is comprised of a ratio between the
infinite-horizon average cost of a policy and the entropy of its long-term
state occupancy measure. The OIR ensures that no matter how many trajectories
an RL agent traverses or how well it learns to minimize cost, it maintains a
healthy skepticism about its environment, in that it defines an optimal policy
which …

arxiv information policy search