Visual captions: Using large language models to augment video conferences with dynamic visuals

Recent advances in video conferencing have significantly improved remote video communication through features like live captioning and noise cancellation. However, there are various situations where dynamic visual augmentation would be useful to better convey complex and nuanced information. For example, when discussing what to order at a Japanese restaurant, your friends could share visuals that would help you feel more confident about ordering the …

alex augmentation augmented reality captioning communication conferences conferencing deep learning dynamic features google hci language language models large language models natural-language understanding noise reality research staff through video video conferencing visuals