A Robust Mixed-Effects Bandit Algorithm for Assessing Mobile Health Interventions

arXiv:2312.06403v3 Announce Type: replace
Abstract: Mobile health leverages personalized, contextually-tailored interventions optimized through bandit and reinforcement learning algorithms. Despite its promise, challenges like participant heterogeneity, nonstationarity, and nonlinearity in rewards hinder algorithm performance. We propose a robust contextual bandit algorithm, termed "DML-TS-NNR", that simultaneously addresses these challenges via (1) modeling the differential reward with user- and time-specific incidental parameters, (2) network cohesion penalties, and (3) debiased machine learning for flexible estimation of baseline rewards. We establish a high-probability regret bound …

abstract algorithm algorithms arxiv challenges cs.lg dml effects health hinder mixed mixed-effects mobile modeling performance personalized reinforcement reinforcement learning replace robust stat.ml through type via