RoBERTa Bytepiece tokenizer - extracting rep positions from sequences.

So I'm trying to extract the RoBERTa representations for particular words in a sentence, for example:

"The cat in the **hat** went to the pond"

Lets say I'm interested in **hat**. I noticed that after tokenizing, if I tokenize "hat" in isolation, its token ID for hat is different from when I tokenize "the cat in the hat went to the pond"...

Essentially, I'm trying to do a study on contextualized word-reps, but if the rep for **hat** is different …

languagetechnology roberta