Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning

Inspired by human conscious planning, we propose Skipper, a model-based reinforcement learning agent utilizing spatio-temporal abstractions to generalize learned skills in novel situations. It automatically decomposes the given task into smaller, more manageable subtasks, and hence enables sparse decision-making and focused computation on the relevant parts of the environment. This relies on the extraction of an abstracted proxy problem represented as a directed graph, in which vertices and edges are learned end-to-end from hindsight. Our theoretical analyses provide performance guarantees …

abstractions agent computation consciousness cs.ai cs.lg decision environment human making novel planning reinforcement reinforcement learning skills temporal the environment