[D] Is there a way of “negative prompting” at fine-tuning time?

Let’s say I’m attempting to fine-tune a pretrained language model, and I’d like to alter its response format. Normally, I’d fine-tune on a bunch of examples of responses in the new format. But doing so would also change the model’s semantic behavior to more closely mimic the type of text present in the SFT examples. Is there a way to fine-tune on an example in the new format, then effectively *negatively* fine-tune on the same text in the finetuning example …

behavior change examples fine-tuning format language language model machinelearning negative normally pretrained language model prompting responses semantic type