Designing LLM Chains by Adapting Techniques from Crowdsourcing Workflows

arXiv:2312.11681v3 Announce Type: replace-cross
Abstract: LLM chains enable complex tasks by decomposing work into a sequence of subtasks. Similarly, the more established techniques of crowdsourcing workflows decompose complex tasks into smaller tasks for human crowdworkers. Chains address LLM errors analogously to the way crowdsourcing workflows address human error. To characterize opportunities for LLM chaining, we survey 107 papers across the crowdsourcing and chaining literature to construct a design space for chain development. The design space covers a designer's objectives and …

abstract arxiv crowdsourcing cs.ai cs.cl cs.hc designing error errors human llm opportunities tasks the way type work workflows