Gaussian Process-Gated Hierarchical Mixtures of Experts

arXiv:2302.04947v2 Announce Type: replace
Abstract: In this paper, we propose novel Gaussian process-gated hierarchical mixtures of experts (GPHMEs). Unlike other mixtures of experts with gating models linear in the input, our model employs gating functions built with Gaussian processes (GPs). These processes are based on random features that are non-linear functions of the inputs. Furthermore, the experts in our model are also constructed with GPs. The optimization of the GPHMEs is performed by variational inference. The proposed GPHMEs have several …

abstract arxiv cs.lg experts features functions gaussian processes gps hierarchical inputs linear non-linear novel paper process processes random stat.ml type