What you say is true, but it still makes for a steady and huge decline in vote share. Brown lost a further 6.2% in 2010 meaning between now and 1997 New Labour have hemorrhaged 25.2% vote share. Milliband only managing to pull it back by 1.5% last year even after 5 years of brutal Tory cuts. I agree that simply saying 4 million votes is sophistry but whichever way dress it up this is a huge decline in votes.