Toxi-Phi: Training A Model To Forget Its Alignment With 500 Rows of Data

I knew going into this experiment that the dataset would be effective just based on prior research I have seen. I had no idea exactly how effective it could be though. There is no point to align a model for safety purposes, you can remove hundreds of thousands of rows of alignment training with 500 rows.

I am not releasing or uploading the model in any way. You can see the video of my experimentations with the dataset here: [https://youtu.be/ZQJjCGJuVSA](https://youtu.be/ZQJjCGJuVSA)

alignment data dataset experiment machinelearningnews phi prior research safety training