Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset

arXiv:2403.09029v1 Announce Type: cross
Abstract: Using vision-language models (VLMs) in web development presents a promising strategy to increase efficiency and unblock no-code solutions: by providing a screenshot or a sketch of a UI, a VLM could generate the code to reproduce it, for instance in a language like HTML. Despite the advancements in VLMs for various tasks, the specific challenge of converting a screenshot into a corresponding HTML has been minimally explored. We posit that this is mainly due to …

abstract arxiv code conversion cs.ai cs.cv cs.hc dataset development efficiency generate html instance language language models no-code solutions strategy type vision vision-language models vlm vlms web web development