You actually make a very salient point. The one situation in which optimal virulence fails to apply (at least at first) is the one in which a new pathogen is introduced extremely rapidly to a never previously exposed population. e.g. introduction of an alien bacteria or sudden re-introduction of a virus or bacterium that humans have not been exposed to in thousands of years. In that scenario the death toll from circulation of a highly lethal and virulent microorganism could be very high indeed. That said the speed at which microorganisms mutate and evolve combined with other factors which would limit its spread (large population decreases, decreases contact points among humans, which necessarily reduces transmission) still suggest to me even in that scenario disease would be self limiting and a mass death event unlikely after a very scary first week or two, month or two, or year or two.