A lot of this is dependent on who won and who lost.
Yes, the Southerners were traitors, in that they took up arms against their own country and declared a chunk of it to be separated from the whole. They may have recognized the CSA to be a separate country, but the USA didn’t. They got branded with the “traitor” label when they seceded, and that continued after they surrendered and attempted to rejoin the Union.
I’m sure the George III’s UK considered Americans to be traitors after the Declaration of Independence, too. However, we won. When you win, you don’t care what the other side calls you. You’re not part of that country anymore.