[D] Does the vocabulary size really affect the size of textual LLMs?

Is the embedding matrix sizeable compared to the other components of the transformer?

If not, then why GPT models are relying on a 30K vocab size?

components embedding gpt gpt models llms machinelearning matrix textual transformer