all AI news
EulerFormer: Sequential User Behavior Modeling with Complex Vector Attention
March 27, 2024, 4:42 a.m. | Zhen Tian, Wayne Xin Zhao, Changwang Zhang, Xin Zhao, Zhongrui Ma, Ji-Rong Wen
cs.LG updates on arXiv.org arxiv.org
Abstract: To capture user preference, transformer models have been widely applied to model sequential user behavior data. The core of transformer architecture lies in the self-attention mechanism, which computes the pairwise attention scores in a sequence. Due to the permutation-equivariant nature, positional encoding is used to enhance the attention between token representations. In this setting, the pairwise attention scores can be derived by both semantic difference and positional difference. However, prior studies often model the two …
abstract architecture arxiv attention behavior core cs.ir cs.lg data encoding lies modeling nature positional encoding self-attention transformer transformer architecture transformer models type vector
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Senior Machine Learning Engineer
@ GPTZero | Toronto, Canada
ML/AI Engineer / NLP Expert - Custom LLM Development (x/f/m)
@ HelloBetter | Remote
Doctoral Researcher (m/f/div) in Automated Processing of Bioimages
@ Leibniz Institute for Natural Product Research and Infection Biology (Leibniz-HKI) | Jena
Director, Global Success Business Intelligence
@ Salesforce | Texas - Austin
Deep Learning Compiler Engineer - MLIR
@ NVIDIA | US, CA, Santa Clara
Commerce Data Engineer (Remote)
@ CrowdStrike | USA TX Remote