UIClip: A Data-driven Model for Assessing User Interface Design

arXiv:2404.12500v1 Announce Type: cross
Abstract: User interface (UI) design is a difficult yet important task for ensuring the usability, accessibility, and aesthetic qualities of applications. In our paper, we develop a machine-learned model, UIClip, for assessing the design quality and visual relevance of a UI given its screenshot and natural language description. To train UIClip, we used a combination of automated crawling, synthetic augmentation, and human ratings to construct a large-scale dataset of UIs, collated by description and ranked by …

abstract accessibility applications arxiv cs.cl cs.cv cs.hc data data-driven design language machine natural natural language paper quality type usability visual