DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers

arXiv:2310.03686v2 Announce Type: replace
Abstract: In recent years, many interpretability methods have been proposed to help interpret the internal states of Transformer-models, at different levels of precision and complexity. Here, to analyze encoder-decoder Transformers, we propose a simple, new method: DecoderLens. Inspired by the LogitLens (for decoder-only Transformers), this method involves allowing the decoder to cross-attend representations of intermediate encoder layers instead of using the final encoder output, as is normally done in encoder-decoder models. The method thus maps previously …

abstract analyze arxiv complexity cs.cl decoder encoder encoder-decoder interpretability interpretation precision simple the decoder transformer transformers type