Not Your Typical Tuesday
Tom Siu, CISO, Case Western Reserve University
This is not the typical Tuesday for me, but it will be a fun one for certain. Today is September 29, 2020, and it is D-Day.
Debate Day.
Background
In last week of July, the CIO and I were informed that Case Western Reserve University (CWRU) and the Cleveland Clinic Foundation (CCF) were going to host the first 2020 Presidential Debate. Due to COVID-19 impacts, the University of Notre Dame had canceled their plan to host the event. Normally, universities that host the debate event have a year to plan and prepare, but we had 6 weeks to get ready. We had to get started on a plan to build a system to support the debate participants (the campaigns and the media) and also defend our own infrastructure from expected attackers that would like to disrupt the event. We began an all out sprint to get the requirements and execution plan going. Fast forward to D-1, we have been in full debate defense operations since the previous Thursday.
Awake
Since COVID-19 operations have had us working from home, I generally wake up early and go for a morning walk. But today I’m awake at 5:49 AM, long before my alarm is set to go at 7:15 AM. The Debate has a couple of us staying in a hotel near the event site, and I’m in the Glidden House, a historic hotel on the CWRU campus. It’s going to be an eventful day so I start my morning with prayers. I don’t always pray in the morning, but today I recite the first and third verses of a Charles Wesley hymn, “Forth in Thy Name, O Lord, I go.” Head is clear, thoughts focused on the day.
I’m up waaaay early, so I decide to put in a power walk around the campus and listen to a podcast. It is cool and clear, and I take a few laps around the athletic field surrounded by the North Residential Village.
I generally listen to one or both of these podcasts on my morning walk: The Briefing by Albert Mohler, or Truth for Life with Alistair Begg. Both are about 20–30 minutes and they help me focus for the day. CISOs have tons of distractions, and getting physical, mental, and spiritual exercise does help. Today’s podcast, The Briefing covers the topic of The Debate, and toward the end of the first section, Dr. Mohler discusses the background of the Case Western Reserve University, and the Connecticut Western Reserve. Hey, that’s us! When I get back to my hotel room, I check my emails and slack messages, then suit up and head over to the Samson Pavilion.
The First Watch
As I noted earlier, we’ve been in debate operations since Thursday, D-minus 5. We have a “Team of Teams” (based on the book by Gen. Stanley McChrystal) of combined leaders and staff from CWRU, CCF and the Committee for Presidential Debates (CPD). We were standing a shift rotation to ensure there was cyber decision-making authority on the debate site, but also did not burn anybody out. We fully staffed the onsite operations with both leadership and analysts starting yesterday, and today I have the first watch. The cyber and IT operations must be in the facility where the debate is held, and therefore we all had to pass COVID-19 screens to be inside, like everybody else in attendance (the two candidates included). The risk that some staff would either test positive, or be notified that they were exposed to someone with the virus, meant we had to plan for personnel redundancy, so we had setup a staffing depth chart with three layers from both teams. Everybody had to be versant in the tools and monitoring, as well as the procedures established for incident response. Some of our COVID test results did not arrive until 9:00 PM Monday night, which added just a little more stress, and I was able to issue the “negative” test armbands to our staff last night to all but one person. This last person called me with his negative test result, and fortunately we were able to get his armband to him.
Getting into the facility was a classic “moat and castle” model. A double fence perimeter was established around the Samson Pavilion, with entry checkpoints manned by police and Secret Service personnel. We all had to wear face masks. The Clinic medical folks preferred us to wear those paper medical masks vs. our custom cloth ones (with the CWRU logo). I wasn’t going to debate the relative efficacy of cloth vs. paper masks, so I took one from them at the gate. They take your temperature with an IR thermometer, and once you showed your “access credential” on the lanyard around your neck, your COVID test armband, they then check your laptop bag thoroughly and wand-test you with a metal detector. No, my cowboy hat did not have any metal alerts, but my belt buckle sure did! The same procedure is conducted upon entry into the main debate facility, with the exception that the CCF has setup thermal cameras and monitors that can detect a fever at fifty paces. Nobody would ever complain about multifactor authentication if they experienced this screening every time they went to work.
Exercises
Once our team entered our Cyber Operations Center, a CWRU classroom setup for polite social distancing, we ran several exercises to clarify our incident response process. We had already run numerous table-top scenarios during the month of September, and I wanted to ensure we were in tune with the Network team/HelpDesk personnel in the room next door. Since this was basically a one-time event, we didn’t need too much sophistication in our communications processes. If we had a problem on the network, we would just walk next door and work it out with them. We also had an open conference call going all day since D-minus 5, which is how we maintained continuity of communications with the respective SOC teams from CWRU and CCF. We had the classroom projector running with our status board, and a large screen television where we could watch the news broadcasts.
The CWRU and CCF cyber security team members did know each other through our shared membership in the Northeast Ohio Cyber Consortium since 2015. However, this was our first joint operation. We had been planning and developing procedures and communications protocols with them using the National Incident Management System (NIMS) from FEMA. CISOs often have a stakeholder role in campus Business Continuity Planning. Knowing these FEMA standards pays multiple dividends for us in clarifying roles and responsibilities in cyber and physical security events such as this.
In the afternoon, I conducted a shift change as the debate “night crew” arrived. Our law enforcement colleagues arrived, and I passed my watch on to my relief, the CISO from the Cleveland Clinic. So far, all preparations are at the ready and we have had no troubling events that we could not handle. Before leaving, I conducted a “dice poll” of the room to determine who would “win” the debate. With a D-20 die that I use for the Backdoors and Breaches card game I carry around with me, and the 1–10 being for one candidate and 11–20 for the other, we had a room full of high rollers.
Debate Night
With the end of my cyber decision-maker watch, I pivoted to law enforcement (LE) liaison for the evening. I walked across the street to join the teams gathered in the Emergency Operations Center (EOC) for the CCF. Unlike the cyber operations center, this room was full of various LE agency representatives, all talking on radios, in a room lined with video screens where it seemed every surveillance camera in Cleveland was available for display. My job, now, was to keep the cyber ops room team appraised of any physical threat and conversely relay additional cyber threat intelligence to the LE team. Knowing that protests had spawned violent riots over the spring and summer throughout the country, we kept a vigilant watch. It is valuable to understand the mission that police and safety forces play in these types of events, and to be certain they know what is going on in the cyber realm as well.
I was hungry by then, and was able to walk over to the CCF cafeteria to grab a really good burger for dinner. Making my way back to to the EOC, I asked the police officer guarding the door if I could go outside the front door and sit on a nearby bench and eat. Initially he let me go, but after I sat down he came out and said they wanted to keep the area clear of people. I had been indoors all day, and as evening approached I wanted to get outdoors for a few minutes and eat. The building we were in was part of a parking garage, so I went to the top floor and out onto the open air to eat. Well, that was a mistake. Men in black with binoculars looking everywhere did not make for a peaceful scene, so I decided to make my way back indoors. I did however photograph this lovely sunset with the Cleveland skyline on my way back inside.
The burger was still warm when I arrived back in the EOC office area, and surprise surprise, I meet up with some LE friends I’ve not seen in awhile. They have their food and we share a meal together and catch up on operations. I would be hard pressed to find a safer venue than this location in all of the Western Reserve tonight.
This article is not about The Debate, but we did get to watch it on television while we continued monitoring cyber events. There was a protest that started near CWRU, but never made its way over to us. We did a round-the-room operations check after the conclusion of the debate, then I packed up my gear and headed back to across the street to the debate hall. I encountered the Fox News anchors on my walk over. Then I saw the cyber team from the CCF leaving the building, and they had already suspended the cyber operations room and were leaving for the night. After all the many weeks of preparations, it was unceremoniously over. We said our good-byes and I went in to collect our radios. I took one last photo of the debate stage before catching a ride back to campus with the CWRU team.
A New Day
The University Police were gracious to us these past few days to give us rides between CWRU and the Samson Pavilion.
It is now Wednesday, September 30, and D-plus 1. I arrived at my hotel at about 1:00 AM. The good news: the news anchors are talking about The Debate. That was our goal. We wanted the media telling the story of The Debate, and not the infrastructure, not the cyber attacks, not the myriad of other distractions that could have been the main story for the day. The team of teams came together for this event, and we prevailed. It was a beautiful day in Cleveland.
It was also my last Tuesday at CISO at CWRU; I’m moving on to a new role.
Having this high-visibility event, with all of the health, safety, and cyber operations to address, seemed like a final exam of sorts. Did the work I’ve done over the past fourteen years truly prepare us for this event?
My team was able to maintain our robust campus security posture while responding to an entirely new series of challenging tasks with little to no preparation time. We successfully navigated the COVID protocols/testing, implemented new and unique security tools, and created an operational team with CWRU and CCF staff to protect the debate from myriad cyber threats. Lastly, the coordination of network, cybersecurity and law enforcement teams was flawless.
Upon reflection, I submit that the work we did in preparation for this event was important, but it rested on the foundation of the lessons learned from many years of incidents and exercises, and processes we’ve implemented at CWRU over the past fourteen years. Seldom does a CISO get a Tuesday like this one!
I send special thanks to my colleagues at CWRU for all their years of “blood, toil, tears, and sweat!”
