all AI news
Transformers changing embedding size (pytorch)
Aug. 10, 2022, 5:58 p.m. | /u/Silly_Ad_4008
Natural Language Processing www.reddit.com
(model.shared means embedding layer (T5 Transformer))
model.shared = new_emb
model.lm_head = new_head
and
model.shared.weight = new_emb.weight
model.lm_head.weight = new_head.weight
The reason that i am asking this question is:
When i use both, i get different loss values (cross-validation loss)
Loss for code piece 1: [https://i.stack.imgur.com/D0sz7.png](https://i.stack.imgur.com/D0sz7.png)
Loss for code piece 2:[https://i.stack.imgur.com/FvhMW.png](https://i.stack.imgur.com/FvhMW.png)
More from www.reddit.com / Natural Language Processing
Which NLP-master programs in Europe are more cs-leaning?
4 days, 23 hours ago |
www.reddit.com
What do you think is the state of the art technique for matching a piece …
6 days, 21 hours ago |
www.reddit.com
AI-proof language-related jobs in the United States?
1 week, 2 days ago |
www.reddit.com
Jobs in AI, ML, Big Data
Lead Developer (AI)
@ Cere Network | San Francisco, US
Research Engineer
@ Allora Labs | Remote
Ecosystem Manager
@ Allora Labs | Remote
Founding AI Engineer, Agents
@ Occam AI | New York
AI Engineer Intern, Agents
@ Occam AI | US
AI Research Scientist
@ Vara | Berlin, Germany and Remote