For a moment there, it looked as though the antagonist of The Walking Dead's fourth season might be pneumonia. The camp had successfully integrated the Woodbury survivors and was finally beginning to make progress in their penitentiary refuge when questionable farming practices unleashed a minor contagion to complement the full-blown plague outside the prison walls. The infected could ride out the virus itself, the characters decided, if only they had enough antibiotics to keep the symptoms from suffocating them to death. So out go the camp heavyweights, scavenging for whatever pharmaceuticals they can find, unless Dr. Bob stumbles upon a bottle of Dewers first, in which case… yawn.

Not that I was at all nostalgic for season three. The Woodbury plot started out promising, but dragged on entirely too long, and without the sinew needed to hold it together. Yet, what the show needed least of all was more fetch plots: characters running to point A, only to find that they can’t get what they came for until they’ve sidetracked to point B, then back to A, and so on. That was a mainstay of season 1, and I had no enthusiasm for a reprise.

Leading up to this year’s mid-season finale, though, the show wrapped up the virus plot with merciful dispatch and followed with a two-episode detour that counter-intuitively reasserted a potential for greatness with which the show has long flirted, but rarely attained.

Two of the aforementioned heavyweights, plus Dr. Bob.

That may seem an odd thing to say. After all, The Walking Dead is AMC’s most popular show. This season’s premiere drew in more than 16 million viewers, more than the combined peak ratings for Mad Men and Breaking Bad. Like those prestige shows, The Walking Dead is marked by fine acting, a mostly consistent tone, and benchmark production values that rival its premium cable competition. It even outdoes most recent big budget horror movies with its inventive treatment of the titular Walkers’ grossly pliable bodies, for which the show has won its only two Emmys.

What makes The Walking Dead perhaps the most consistently frustrating show on prime time are the nuts and bolts of its storytelling. The plots—like season 2’s doomed search for Sophia—work best when you take the long view, but the narrative beats from which they’re composed are often clunky and disjointed. Getting from plot-point to plot-point often requires characters to behave erratically. Psychological change happens suddenly, with precious little development.

That, in part, explains why the post-episode discussions in The Talking Dead are so critical to the show’s success. Buoyed up by Chris Hardwick’s dauntless enthusiasm for all things nerd, they encourage viewers to retell the story to themselves. That lets them dwell on the high points, suture together the connective tissue of the overarching plots, and fill in the show’s psychological ambiguities to their own satisfaction.

If all that fails to sell the shifts the writers need in order to progress the plot, there’s always a final resort in the extremity of the situation. If a character’s behavior doesn’t quite square with how we’ve understood them up to that point, well, the apocalypse makes everyone a tad unpredictable, now doesn’t it?

Or does it? It’s so far outside the bounds of common experience that we don’t really know. Every time the story relies on apocalyptic stress to change a character, it makes it a little harder relate. That might not have been a problem except that so much of The Walking Dead is about how people relate (or don’t relate) to one another. Its central conceit is not so much the bodies milling about the landscape, as the premise that, stripped of the civilizing crutches of modernity, we’re forced to struggle against ourselves if we want to remain connected to people. That struggle grows a little harder to believe every time the audience can’t quite make sense of a sudden change of mind or left-field resolution.

Which is precisely why the two most recent episodes—“Live Bait” and, to a lesser extent, “Dead Weight”—took me so much by surprise. By the 2nd half of season 3, I had begun to find the Governor more tiresome than compelling. It’s easy enough to imagine how a leader faced with a constant state of emergency would develop a markedly draconian bent, but the more grotesque turns in his character never really added up.

How, for example, were we to square his disdain for the “Biters” with the literally closeted devotion he showed to his reanimated daughter? Wind-down periods with a glass of Scotch and a wall of Biter heads suggested that he was simply deranged: a plausible explanation, but one that turned him into a mustache-twirling caricature, rather than a three-dimensional foil. There’s nothing inherently wrong with a two-dimensional character—the novelist E.M. Forester argued that a well-constructed plot could hardly move without them—but the point is to let them do their job, then usher them back out again. The Governor never quite managed a third dimension, yet the show stubbornly refused to kill him.

Might be time for a change.

He first resurfaced at the end of episode 5 (“Internment”), the camera pulling back on a peaceful father-son moment to show him watching from the woods. I was nonplussed, but the next episode betrayed the expectation set by that menacing reappearance. Right away, it set itself to the task of stripping him down to nothing. By the title sequence, he was ready to start working on that third dimension.

Broadly, “Live Bait” is structured as though its central plot were about the Governor were building a new family, but it’s more effective as the story of a man rebuilding himself. Its very much a bottle episode, honing in on a single character and a self-contained plot to the exclusion of nearly every other plotline the show has followed up to this point. That narrowness of focus is very much to its advantage. Its new setting (an apartment building occupied by a family that’s sheltered itself to the point of obliviousness) allows the episode to recapitulate the essential horror of the situation. In one scene, the Governor (now calling himself Brian) searches an apartment for a backgammon set, only to discover its previous tenant in the bathtub, still biting after a badly botched suicide attempt. Barely a threat at all, the ghoul invites us to reconnect the Biters to people as they were before the world fell apart and their mundane lives were shattered.

More importantly, though, the episode makes Brian relatable. In rapid order, he goes from being the impenetrable villain of season 3, to a man reawakening to himself after (literal) months and (figurative) years of wandering. Asked what he’s been up to since the Biter outbreak, he says, “Surviving,” as though Woodbury were just another struggle to get to where he is now. By giving a false name, he’s disavowing an identity that was—let’s be honest—gratuitous.

Remarkably, that transition works, not least of all, perhaps, because what we’ve seen of his past has, for us, the same nightmare quality that it’s supposed to have for him. It works, moreover, as its own story, and I found myself more involved than in any plotline the show has offered so far. The story of a tyrant becoming a man, of a monster becoming relatable: it’s a good story. Good enough, in fact, that we could leave the other camp to their prison base and follow Brian for a while; maybe even indefinitely.

For better or worse, though, the show is anchored to Rick Grimes’ uncertain band of survivors, and “Dead Weight” began the work of dovetailing the two plots back together. That episode begins with the Governor’s former lieutenant Martinez pulling Brian and his surrogate daughter, Meghan, from one of the mass grave-style pits they used to trap Biters outside Woodbury. That image prefigures the resurrection of the Governor identity, but for a while, at least, you could almost believe that the episode was about Brian overcoming the pull of his draconian past.

That’s the plot suggested by another pre-title image, Brian teaching Meghan chess while he hangs laundry on a line strung between a camper and a tank. The tension that presents—between the makeshift home life on one hand, and the protectiveness run amok on the other—is far more resonant that the Governor’s rebirth from the Biter pit. In terms of plot, though, its real significance was not the internal struggle—by the end of the episode, that would already largely be decided. Rather, as preview scenes from the mid-season finale suggest, the real point was the introduction of that tank, which the reborn Governor can now use to threaten the Grimes camp.

Thus begins the rapid unravelling of everything “Live Bait” achieved. The Governor may have updated his motive, but the months of wandering haven’t taught him any new methods. Martinez wants to “share the crown,” but that’s not how dictatorships work. Nor is he the strongest arm in the camp anymore, what with a tank driver among their ranks. Competition is eliminated; new alliances are formed. The Governor even finds time to start a new aquarium at the bottom of the lake. Unless you’re willing to content yourself with a deranged villain, the steps that get us to this point don’t really parse, but that doesn’t matter so long as the plot keeps rolling.

Once again, the show is playing chess with the characters’ personalities to orchestrate a dramatic confrontation. That likely won’t hurt its ratings: there are, after all, too many appealing elements to let a false start like that dissuade viewers. For almost two full episodes, though, The Walking Dead revealed the character-driven kind of show it might have been, and it was good.

Steamrolled by a plot device.