What evidence other than the quote from James Madison suggests other Southern States would’ve pulled out without the use of the Electoral College for the executive branch?
The entire Three-Fifths Compromise is all the evidence you need. It is not a coincidence that the electoral college is modeled exactly after the Three-Fifths Compromise in that the number of electors a state has is precisely the number of congresspersons that state has. I fail to see how you can acknowledge that the Three-Fifths compromise was a concession to the South but the electoral college, whose composition is arrived at in the exact same way, was not. The exact same electoral calculus was applied in both places and, in both places, it allowed for the complete disenfranchisement of black people and the inflated representation of Southern slaveholders.
Were Northern statesmen really so principled that they would allow Southern whites to have a greater political say than they had all for the sake of having an escape hatch against a theoretical demagogue? I sincerely doubt it, and logic plus available evidence points toward the electoral college being a concession to Southern slaveholders. Besides, preventing tyranny and demagoguery was the whole purpose of a system of checks and balances in the first place, which was already built into the three branches of government, and that’s what they wanted to implement: a democratic republic with a system of checks and balances. They wanted to create a union of states with this type of government in place, and they had to make concessions to get there, even if it meant the systematic disenfranchisement of black people. That’s why we have the electoral college.