Linear algebra with transformers. (arXiv:2112.01898v2 [cs.LG] UPDATED)

Transformers can learn to perform numerical computations from examples only.
I study nine problems of linear algebra, from basic matrix operations to
eigenvalue decomposition and inversion, and introduce and discuss four encoding
schemes to represent real numbers. On all problems, transformers trained on
sets of random matrices achieve high accuracies (over 90%). The models are
robust to noise, and can generalize out of their training distribution. In
particular, models trained to predict Laplace-distributed eigenvalues
generalize to different classes of matrices: …

algebra arxiv linear linear algebra transformers