Assessing the Unitary RNN as an End-to-End Compositional Model of Syntax. (arXiv:2208.05719v1 [cs.CL])

We show that both an LSTM and a unitary-evolution recurrent neural network
(URN) can achieve encouraging accuracy on two types of syntactic patterns:
context-free long distance agreement, and mildly context-sensitive cross serial
dependencies. This work extends recent experiments on deeply nested
context-free long distance dependencies, with similar results. URNs differ from
LSTMs in that they avoid non-linear activation functions, and they apply matrix
multiplication to word embeddings encoded as unitary matrices. This permits
them to retain all information in the …

arxiv rnn syntax