OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning

arXiv:2403.13351v1 Announce Type: new
Abstract: Redundancy is a persistent challenge in Capsule Networks (CapsNet),leading to high computational costs and parameter counts. Although previous works have introduced pruning after the initial capsule layer, dynamic routing's fully connected nature and non-orthogonal weight matrices reintroduce redundancy in deeper layers. Besides, dynamic routing requires iterating to converge, further increasing computational demands. In this paper, we propose an Orthogonal Capsule Network (OrthCaps) to reduce redundancy, improve routing performance and decrease parameter counts. Firstly, an efficient …

abstract arxiv attention capsule challenge computational costs cs.cv dynamic layer nature networks pruning redundancy routing type