How does "clip-vit-large-patch14" aggregate the text sequence representation into a singular vector that represent the entire sequence? There is no [CLS] token, but [SOT] and [EOT] tokens. [Research]

Hello guys,

I have the following question:
How does "clip-vit-large-patch14" aggregate the text sequence representation into a singular vector that represents the entire sequence? There is no \[CLS\] token, but \[SOT\] and \[EOT\] tokens.

When I use the CLIP Text Encoder and extract the pooler\_output.. how exactly is this vector created? Is the \[SOT\] token used as \[CLS\] token? Or does a pooling operation take place?

\[Research\]

Best regards,
Tom

clip hello machinelearning question representation research singular text token tokens vector vit