Handling Reward Misspecification in the Presence of Expectation Mismatch

arXiv:2404.08791v1 Announce Type: cross
Abstract: Detecting and handling misspecified objectives, such as reward functions, has been widely recognized as one of the central challenges within the domain of Artificial Intelligence (AI) safety research. However, even with the recognition of the importance of this problem, we are unaware of any works that attempt to provide a clear definition for what constitutes (a) misspecified objectives and (b) successfully resolving such misspecifications. In this work, we use the theory of mind, i.e., the …

abstract artificial artificial intelligence arxiv challenges cs.ai cs.lg domain functions however importance intelligence recognition research safety type