all AI news
Quoting Timothy B Lee
July 28, 2023, 11:30 a.m. |
Simon Willison's Weblog simonwillison.net
You can think of the attention mechanism as a matchmaking service for words. Each word makes a checklist (called a query vector) describing the characteristics of words it is looking for. Each word also makes a checklist (called a key vector) describing its own characteristics. The network compares each key vector to each query vector (by computing a dot product) to find the words that are the best match. Once it finds a match, it transfers information [the value vector] …
ai attention checklist generativeai llms network query service think vector word words
More from simonwillison.net / Simon Willison's Weblog
We can have a different web
1 day, 20 hours ago |
simonwillison.net
Introducing the Claude Team plan and iOS app
2 days, 6 hours ago |
simonwillison.net
Jobs in AI, ML, Big Data
AI Research Scientist
@ Vara | Berlin, Germany and Remote
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Data Science Analyst
@ Mayo Clinic | AZ, United States
Sr. Data Scientist (Network Engineering)
@ SpaceX | Redmond, WA