How to add additional module to BERT architecture, then load the original weight and use it

I aim to

1. Add an additional module to BERT architecture (huggingface’s transformers)
2. Load the BERT’s weight to the BERT model with new architecture
3. Then use BERT directly or continue train BERT

I’m very confused how to do it. Since we usually use `from_pretrained` directly to load the model (both weight and architecture) from huggingface.

In more detail, I’m working on both [prajjwal1/bert-tiny](https://huggingface.co/prajjwal1/bert-tiny) and [bert-base-uncased](https://huggingface.co/bert-base-uncased).

architecture bert languagetechnology