Collective Constitutional AI: Aligning a Language Model with Public Input

arXiv:2406.07814v1 Announce Type: cross
Abstract: There is growing consensus that language model (LM) developers should not be the sole deciders of LM behavior, creating a need for methods that enable the broader public to collectively shape the behavior of LM systems that affect them. To address this need, we present Collective Constitutional AI (CCAI): a multi-stage process for sourcing and integrating public input into LMs-from identifying a target population to sourcing principles to training and evaluating a model. We demonstrate …

abstract arxiv behavior collective consensus constitutional ai cs.ai cs.cl cs.hc developers input language language model public shape systems them type