You are not doing it right. You can’t just screw things up and call them broken.
It is all about responsibility indeed, but it doesn’t mean that “one person is responsible for the whole system.” Code ownership means that there is someone specific responsible for each and every line of the code. It’s not some abstract responsibility, it’s a very concrete thing — you can get fired for what you allow in your code. Is it stressful? It should be. Responsibility is stressful, but irresponsibility is not tolerable in any other field of engineering. Why should we bare less responsibility than chemical or mechanical engineers?
If something goes wrong with the code, it’s not necessary a code owner’s responsibility to fix it. It is an owner’s responsibility to make sure the fix doesn’t bring any more problems than it fixes. Because an owner has to have the most expertise over his own code and he often is the only person that can (and most importantly should) predict the implications of changes.
Speaking about code reviews, it is owner’s responsibility to review all the code committed into his zone of responsibility. Because he collects the most expertise. And that’s also how he collects the most expertise. Committing without review is either way unacceptable, but with the code ownership you know whom you definitely should involve.
Key man dependency is just an orthogonal issue. You can’t expect all the team members to be proficient in all of the code base. Sooner or later there will grow clusters of expertise. Some people naturally get to work most with some part of the code rather than the other. Without code ownership these clusters grow uncontrollably. When your key person is leaving the company you just don’t know what exactly you are loosing.
With code ownership you can plan transitions, arrange backups and otherwise manage dependencies predictably. In our company there is usually two co-owners per component and ownership areas overlap extensively over the department. If I get hit by a bus, I know who will replace me for each and every of my components exactly. If two of us get hit by the buses, there will be only a small gap in the ownership scheme when our zones of responsibility coincided.
Code ownership is a tool. Like a drill or a chainsaw. If you’re using it wrong, you should only invest a bit into learning how to use it right and not revert to “the culture of cutting trees with axes.”