Scaling up Discovery of Latent Concepts in Deep NLP Models

Despite the revolution caused by deep NLP models, they remain black boxes, necessitating research to understand their decision-making processes. A recent work by Dalvi et al. (2022) carried out representation analysis through the lens of clustering latent spaces within pre-trained models (PLMs), but that approach is limited to small scale due to the high cost of running Agglomerative hierarchical clustering. This paper studies clustering algorithms in order to scale the discovery of encoded concepts in PLM representations to larger datasets …

analysis black boxes clustering concepts cs.cl decision discovery making nlp nlp models pre-trained models processes representation research scale scaling scaling up small spaces through work