You are aware that the energy doesn’t get eliminated, but transformed?
There would be a much smaller price to pay reversing this local process, reversing the local entropy (at the expense of increasing entropy elsewhere).
I wonder if you have to re-arrange the universe or only the sphere of the universe which was able to witness the events you are undoing.
Anyway, this form of time travel doesn’t reverse the process. You don’t get to interfere with any observable process by this model, unless you allow faster than light locomotion between the two wormhole termini, or if you allow a circular motion of the second terminus into the vicinity of the first one — and there is the possibility that trying to enforce such a circular motion will collapse the wormhole.
But: assuming you can build a pair of such termini and move one side of them, what is keeping you from starting another such pair half the projected travel distance away from your stationary positive terminus, and move the positive second terminus into the vicinity of the end point of your mobile negative terminus, and the second negative terminus at the same speed to the vicinity of your stationary first terminus, so you get no time differential for your second pair of wormholes.
Such a constellation would allow you to do a paradox travel.
To prevent the paradox, at least one of the pairs needs to collapse when it approaches its anti-parallel counterpart into observation radius (which is by necessity greater than the distance traveled by any object passing through the wormhole termini).
There is no problem with an object passing through the terminus along the time axis vector without delay since its emergence would be postponed. Ramming an object through the wormhole against the time axis vector might require huge amounts of energy, possibly obliterating any information around the entry terminal. Possibly retroactively — both your traveler and his history might emerge as just an energy smear.