all AI news
Towards Accelerating Training of Batch Normalization: A Manifold Perspective. (arXiv:2101.02916v2 [cs.LG] UPDATED)
June 6, 2022, 1:11 a.m. | Mingyang Yi
cs.LG updates on arXiv.org arxiv.org
Batch normalization (BN) has become a critical component across diverse deep
neural networks. The network with BN is invariant to positively linear re-scale
transformation, which makes there exist infinite functionally equivalent
networks with different scales of weights. However, optimizing these equivalent
networks with the first-order method such as stochastic gradient descent will
obtain a series of iterates converging to different local optima owing to their
different gradients across training. To obviate this, we propose a quotient
manifold \emph{PSI manifold}, in …
More from arxiv.org / cs.LG updates on arXiv.org
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Cleared Senior Software Engineer, Computer Vision, Federal
@ CCRi | Chantilly, Virginia, United States
Data Analyst - B2C
@ DAZN | Hyderabad, India
Product Marketing Manager - AI Chatbot
@ SendBird | San Mateo, California, United States
Alternance Alternant Ingénieur Développement logiciel temps réel embarqué / computer vision (F/H)
@ Alstom | Villeurbanne, FR
AOT Data Analyst II - Highway Project Delivery
@ State of Vermont | Barre, VT, US