Extinction Risks from AI: Invisible to Science?

arXiv:2403.05540v1 Announce Type: cross
Abstract: In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart's Law as "Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity", and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart's Law holds or not. As our key contribution, we identify a set of conditions that are necessary for …

abstract aim arxiv cs.cy cs.lg existential risks extinction humanity hypothesis law risks science type will