Great work !!! Clapped ...

Per my understanding ORPO can work on SFT tuned model also saving memory of two models loaded in memory. SFT requires some another format of data that sometimes can't be translated to Accepted/Rejected format . What do you think ?