Word2vec/LSTMs and subword tokenization

A data scientist on our team is curious what would happen if we'd use subword tokenization (bert tokenization) as the tokenization step for our conventional models (word2vec, CNNs, LSTMs). The word2vec model is used for recommendation and clustering in addition to serving "just" as the embedding layers of other models. We said we'd try it out.

My own intuition is that it would decrease the quality of the word2vec model, since we want this model specifically to distinguish between things …

bert clustering cnns data data scientist embedding languagetechnology recommendation team tokenization word2vec