Ok as Far as You Go, But Need To Go Step Further
Evolution at all scales does have strong redundancy feature but not for the reason suggested. For example, when you say, “It’s like there’s a cosmic principle that commands the universe to ‘always make more than is necessary.” That would not have any intrinsic value. Rather, value in doing such is only as part of a larger search strategy.
Think of it as water flows. Naturally occurring water is pulled by gravity toward the oceans. It can take numerous routes that result in pooling (e.g., lakes and ponds, or your redundant waste). But eventually even the pooling backs up and forces additional water to flow in different direction to get back on its trajectory toward the oceans via a river or stream (e.g., your trial and error).
Said differently, evolution is not trying to “weed out the less desirable aspects of existence,” rather find what option will best advance evolution further.
In Kevin Kelly’s book, “What Technology Wants,” he calls it the “infinite game.” In my work goes deeper and I call it “maximizing evolvability.” That is the principle — a cosmic prime directive — operating at all evolutionary system scales.
On my Medium publication, A Passion to Evolve, there are numerous articles that explicate this in detail that you might find interesting. The most detailed, big picture discussion is in the article, Macroscopic Evolutionary Paradigm. Some of other shorter articles include, A Message From the Bow of Civilization, Who Controls What Happens with the Prime Directive?