DrawTalking: Building Interactive Worlds by Sketching and Speaking

We introduce DrawTalking, a prototype system enabling an approach that empowers users to build interactive worlds by sketching and speaking. The approach emphasizes user control and flexibility, and gives programming-like capability without requiring code. An early open-ended study shows the mechanics resonate and are applicable to many creative-exploratory use cases, with the potential to inspire and inform research in future natural interfaces for creative exploration and authoring.

build building capability cases code control creative cs.ai cs.cl cs.gr cs.hc enabling exploratory flexibility interactive programming shows speaking study use cases