Efficient Transformer-based Hyper-parameter Optimization for Resource-constrained IoT Environments

arXiv:2403.12237v1 Announce Type: new
Abstract: The hyper-parameter optimization (HPO) process is imperative for finding the best-performing Convolutional Neural Networks (CNNs). The automation process of HPO is characterized by its sizable computational footprint and its lack of transparency; both important factors in a resource-constrained Internet of Things (IoT) environment. In this paper, we address these problems by proposing a novel approach that combines transformer architecture and actor-critic Reinforcement Learning (RL) model, TRL-HPO, equipped with multi-headed attention that enables parallelization and progressive …

abstract arxiv automation cnns computational convolutional neural networks cs.ai cs.lg environment environments internet internet of things iot networks neural networks optimization paper process transformer transparency type