Equilibrium Aggregation: Encoding Sets via Optimization. (arXiv:2202.12795v2 [cs.LG] UPDATED)

Processing sets or other unordered, potentially variable-sized inputs in
neural networks is usually handled by aggregating a number of input tensors
into a single representation. While a number of aggregation methods already
exist from simple sum pooling to multi-head attention, they are limited in
their representational power both from theoretical and empirical perspectives.
On the search of a principally more powerful aggregation strategy, we propose
an optimization-based method called Equilibrium Aggregation. We show that many
existing aggregation methods can be …

aggregation arxiv encoding equilibrium lg optimization