Why reward models are key for alignment

In an era dominated by direct preference optimization and LLM-as-a-judge, why do we still need a model to output only a scalar reward?

alignment direct preference optimization judge key llm optimization