Apple researchers develop AI that can 'see' and understand screen context

- Apple researchers have developed an AI system called ReALM that can understand screen context and ambiguous references, improving interactions with voice assistants.

- ReALM reconstructs the screen using parsed on-screen entities to generate a textual representation, outperforming GPT-4.

- Apple is investing in making Siri more conversant and context-aware through this research.

- However, automated parsing of screens has limitations, especially with complex visual references.

- Apple is catching up in AI research but faces stiff competition from tech …

ai system apple artificial assistants context generate gpt gpt-4 improving interactions investing making realm representation researchers siri textual voice voice assistants