Reinforcement Learning in Categorical Cybernetics

arXiv:2404.02688v1 Announce Type: new
Abstract: We show that several major algorithms of reinforcement learning (RL) fit into the framework of categorical cybernetics, that is to say, parametrised bidirectional processes. We build on our previous work in which we show that value iteration can be represented by precomposition with a certain optic. The outline of the main construction in this paper is: (1) We extend the Bellman operators to parametrised optics that apply to action-value functions and depend on a sample. …

abstract algorithms arxiv build categorical cs.lg cybernetics framework iteration major math.ct optic processes reinforcement reinforcement learning show type value work