Rebuilding After Michael

Lessons Learned from 48 Hours with Telecom Crews

Brendan Carr
10 min readNov 6, 2018

Adrian’s phone kept buzzing. He rolled over in bed on the morning of Wednesday, October 10, to find two dozen messages from friends and neighbors. They all said essentially the same thing:

GET OUT

Adrian had moved to Mexico Beach, Florida, about five years before. His wife, Betsie, is from the Panhandle, and he landed a job as the city clerk. It was an ideal place to end up — white sand beaches, renowned fishing. Teachers and servicemembers fulfilled their lifelong dreams of retiring there along the beach.

That was all before Michael.

At 1:30 PM on October 10, Hurricane Michael landed on Florida’s Gulf Coast. Michael peaked just below a Cat 5 with sustained winds clocked at 155 miles per hour. It ranks as the strongest hurricane to touch down on U.S. soil in nearly 50 years.

Mexico Beach took the direct hit.

Adrian recounts riding out Michael in Mexico Beach

A transplant from Minnesota, Adrian wanted to ride out at least one small-scale hurricane while living in Florida. He’d get some experience with the weather of his new home, and, as a city official and firefighter, he could be first on the scene to help his fellow residents. “Heck of a storm to pick,” said a fellow Floridian.

On Monday, Michael was a tropical storm on its way to becoming a Cat 1 hurricane. By Tuesday, it had been upgraded to a Cat 3, and the National Weather Service said that it might make landfall as a Cat 4. Neighbors told Adrian that he could stay through a Cat 3 with the appropriate precautions, and a number of his friends decided to ride out the storm. Those plans changed as Michael intensified into a nearly Cat 5 storm over Tuesday night and early Wednesday morning. Adrian’s friends called and texted to warn him to get out of town. But Adrian and his wife Betsie, another firefighter, did not see the messages until they woke up later that morning.

By then, it was too late.

The bridges out of town were closed. Adrian and Betsie were stuck.

Just after noon, Adrian texted his mom:

I’m seeking shelter in my bedroom closet.

His mom would not hear from him again for more than a day.

For the next few hours, Michael leveled Mexico Beach.

From the mangled remains of the home he bought two months ago, Jack, a retired teacher and army vet, pointed a few blocks in the distance. It’s where a house was ripped off its foundation and sent tumbling nearly a quarter mile down the road, disintegrating every structure in its path. The city’s water tower — made out of reinforced steel and the city’s tallest structure — bent like a paperclip. The punishing winds and debris were followed by a storm surge of 14 feet or more. Cars and garages were pushed inland with the water, trapping residents in their homes. It was a level of devastation that even veteran first responders told me they had never seen.

Life-or-Death Communications

Heath, a lanky 18-year old, goes to high school in Port St. Joe, just east of Mexico Beach. He’s also a volunteer fire fighter. Heath’s dad helps run the city’s fire department, so Heath grew up around the firehouse.

With the storm approaching, the fire department held a meeting. Would they stay or go? Ten fire fighters chose to stay behind, knowing that some residents would refuse to evacuate. Heath was one of the 10.

Health shows us one of the rooms where 40 people rode out Hurricane Michael

There are two cinderblock rooms in the Port St. Joe firehouse, each the size of a small bedroom. They were just large enough for Heath and the other firefighters to ride out the storm. But as the eye of Hurricane Michael approached, residents made their way to the fire station to seek a stronger shelter. Ultimately, 40 people crammed into the two small rooms.

The cinderblock walls were rated for a Cat 4, “and we pushed them beyond their limits,” Heath said. Staring up at the ceiling, “I could see the roof rolling like waves.” Outside the small room, Heath heard loud banging and crashing sounds. He’d find out a few minutes later that Hurricane Michael ripped the station’s metal garage doors off their frames. The wind sent them sailing through the bay where the department parked their firetrucks before the doors blasted a hole through the back side of the station.

Top left: The open bay where the doors to the Port St. Joe fire station once stood. Top right: Close up of the scrapes where the hurricane ripped the doors from their hinges. Bottom: Forceful winds snap a forest of pines in half.

Emergency communications were hit hard. A number of the cell antennas in Mexico Beach sat on top of the city’s water tower, which was bent to the ground. Heath and the Port St. Joe fire fighters communicated using “talk arounds” — the radio-to-radio, walkie-talkie feature on their handhelds.

The howl of Michael’s winds and the clanking of weaponized debris against the station quieted, eerily so and long before the storm should have ended. At that moment, the firefighters knew they were in the eye of the hurricane.

They soon heard a banging on the door and found a neighbor soaked and panicked. His wife was trapped in their truck, which had been washed into a tree from the storm surge, and she wasn’t breathing.

Heath waded through the chest-deep storm surge to reach the woman. He guessed she was 50 pounds heavier than he was. One look at Heath, and she said, “you’re not going to be able to lift me.” “I sure showed her,” Heath said. He carried her up a flight of stairs into the safety of a nearby house. “I’m not sure I could do it again.”

Rebuilding, Foot-by-Foot

In Hurricane Michael’s wake, infrastructure first responders rushed to restore power and communications. More than 7,000 power company contractors and crews cleared downed wires and placed pole after pole into the ground. Wireless carriers stationed cells-on-wheels (COWs) in the surrounding areas. The COWs — trucks with wireless antennas and satellite dishes for backhaul — provided cell coverage in places where cell towers were down or wrecked antennas and wiring made them inoperable.

With Josh and Blake

Within a day of Michael passing, telecom crews were on the ground: trenching conduit, pulling aerial cables, digging new hand holes, splicing fiber, and running directional bores to extend lines.

Josh was one of the first on the scene. His first shift went 50 hours straight. By the time I joined his team last week, he had been in Mexico Beach and Panama City for 19 days. Eric, the Navy veteran who runs the company’s emergency response operation, noted that the crews are eager to get the job done. In fact, he sometimes has trouble getting Josh’s team and others to stick to 12-hours-on, 12-hours-off shifts (and to rest when they’re off).

On the day I spent with Josh, his team was working to reconnect a medical clinic that had lost service. Their first challenge was to find the break in service by testing each fiber strand. That task fell to Blake, one of the crew’s fiber splicers.

Blake has four kids back home in Mississippi and he’s been away for almost the whole 20 day stretch since Hurricane Michael.

Blake’s work requires patience. Each black wire that hangs from a utility pole can contain hundreds of glass fibers as thin as a hair. Cutting, testing, running over to where they guess the latest break in the line is — this is how fiber crews rebuild after a hurricane, foot-by-foot.

The crew thought it found the break and worked to replace the damaged line. The team got up in bucket trucks near where the signal failed and ran replacement fiber from a giant spool up to the top of the utility pole. At the end of this fiber run, the line dove underground.

So the team grabbed a couple of shovels and dug space between the company’s concrete hand hole and the utility pole, where the new fiber was hung. Within a few hours, the fiber run was live.

By then, the sun had set, and it was unusually dark out for being in a city. Electricity was patchy, and with the nightly curfew approaching, fewer cars were speeding down the roadway. Although the main fiber run was live, the spur that connected the run to the medical clinic still wasn’t working. Josh was determined not to let the day close without one more small win.

Josh rattled the clinic’s front door. Locked. Not giving up, he walked around the perimeter and found a back door ajar, with a tech for another company working on the building’s interior wiring. He opened the utility closet, and with the consultant’s help, plugged an Ethernet cable into the router, and patched the router into the fiber he had worked on all day.

The clinic was back online. High fives, and his 16-hour shift was over.

It was about 10 PM, dinner time for Josh and his team. Josh’s two-year-old daughter back in Alabama Facetimed him. There were big smiles between bites of the burger. Afterwards, he dropped us off and, about seven hours later, he would be back on the fiber line.

Lessons from the Panhandle

I accepted the invitation to spend 48 hours with these telecom crews to see firsthand the work it takes to rebuild a communications network after a storm like Michael. It’s tough work, and the men and women on these teams are giving it their all.

It takes countless small wins to get the job done — locating a single cut line within a mile-long run, trenching fiber by hand, splicing thousands of fibers one strand at a time, getting a single customer back online and then building on that to connect an entire block. I am grateful for the hard work these crews are putting in. And while my two days in Florida provided only a glimpse into the broader recovery effort, I wanted to share a few impressions.

Even with significant communications outages in the wake of Hurricane Michael, many portions of the wireline network, including fiber and cable plant, showed notable resiliency. While in Mexico Beach, for instance, the crew showed me a fiber line that sat yards under the storm surge, yet remained operational.

In many cases, however, lines that weathered the storm were cut (sometimes multiple times) during the recovery and restoration effort. Line cuts during storm restoration are nothing new — it’s why we often see ups and downs in the FCC’s daily outage reports following a storm. But there are lessons we should learn from this experience.

One is that utility first responders need to coordinate more closely. More than 7,000 power company crews and contractors worked around the clock to restore power, and this meant cutting, pulling, and replacing thousands of utility poles. Unfortunately, their work resulted in a significant number of cuts to fiber and other communications lines. In fact, one fiber company reported 37 cuts in the first few days following the storm. Sometimes, lines were cut clean off damaged poles when they could have been detached and put on the ground or left in place. In fact, we saw a few instances where power companies were able to remove the bottom half of a pole, but leave the top portion in place, held up by the intact communications lines.

Example of the top half of a damaged pole left in place with communications lines still intact

Just last week, Chairman Pai appointed first responders and industry representatives to a working group on disaster response and recovery to encourage inter-industry planning. That’s the right step based on what I saw in Mexico Beach.

A significant number of Floridians lost wireless service during and after the storm. Hurricane Michael’s nearly unprecedented winds bent towers, knocked antennas out of alignment, and ripped communications lines off poles. The loss of wired connections — including the backhaul needed to connect a tower to the network — exacerbated the problem on the wireless side.

There are lessons to be learned here, too. For instance, the wireless resiliency framework the FCC put in place in 2016 recognizes that a carrier can roam on another wireless provider’s network when the carrier’s own network has become inoperable. This can help ensure that consumers do not lose service. The reports I heard on the ground were that this type of roaming worked well when carriers exercised their roaming rights. But I heard conflicting reports about how quickly some providers implemented roaming. There should be greater awareness about resiliency roaming and planning before the next disaster.

The Mexico Beach officials I spoke with also talked positively about the carriers that quickly deployed satellite-based COWs. These are the mobile trucks that provide cellular service by using satellite backhaul, thus avoiding the damaged terrestrial network. The officials indicated that some carriers brought in more COWs than others in the first few hours. The same officials expressed gratitude for the wireless charging stations and other facilities that providers deployed in the area.

Above all, I came away with an immense respect and appreciation for the crews on the ground. They have been working to exhaustion to stand up connectivity. Their work requires patience in the face of setbacks. It means being hundreds of miles away from their families — sometimes for months at a time. I’m thankful for their service and the critical role they are playing in the Panhandle’s recovery.

--

--