Fixing the “most terrible website on Earth”
We are now a couple of months after launching the new SI.com, finally with some time to reflect on the past year and share a little on how we arrived here, today.
The situation (not the Jersey shore guy)
Coming into Sports Illustrated back in May 2015, I spent the first week thinking “Holy cow! What is going on here?”
The state of the main website was sub-optimal to understate it. The engineering and product team had been decimated and those that remained were over committed. The relationship with Edit was fractured and wrought with distrust and we had an audience that constantly complained about the state of the site performance, crashes and more (I can only thank them for their loyalty through this dark time.).
While fixes went out everyday, the site was extremely difficult to maintain with a hodgepodge of integrations, way to many contributions from way to many long lost developers, little product direction and built on a bad foundation. A site where an errant ad introducing a single Javascript error could disable the entire site. It was a game of whack-a-mole.
We were playing defense and we were losing the game.
But there were was a silver lining…
The team
What remained of the engineering team had a incredible culture led by a couple of extremely bright and talented web engineers, Harry Hope and Aaron Marasco. They seemed to be keeping the site running through sheer force of will. Better yet, they also had ideas how to fix it — great ones.
We had a new Head of Product, Krys Krycinski who had started a week before me. I immediately liked this guy. Both he and I shared a singular vision for the digital Sports group. Bring back the “Illustrated” part of SI and build the best damn products we could. Nothing less than world class.
We started to rebuild the team around the remaining folks, picking up a deeply talented team of Engineers from HowStuffWorks when InfoSpace shut down their engineering team. They brought Dev Ops and Big Data skills that would ultimately allow us to work mostly as an autonomously unit. This allowed us to operate like a start up, building great products with the ability to pivot rapidly if needed.
The Road Map
The product roadmap was so long it looked more like a wish list than a road map. The only way forward was to reset. Krys and I threw it out. What we knew in our heads, was that there was a redesign in our future. Not only that but we had outgrown our 3rd party hosting provider. That meant migration. We needed the front end to be able to scale, both for high traffic but for developer output as well. Read re-architecture.
We then went back to the stakeholders and asked them, what was really important and would it give us revenue? If it wasn’t important, didn’t make us money and didn’t push us towards a better, more stable platform, we weren’t going to do it.
Additionally we did something new; we added Engineering goals to the road map. We agreed that wherever possible, every step we took, would take us towards a completely new product…
…a shiny new SI.com.
Given that we had to build up trust in our ability to deliver, a full blown, green-lit redesign was a pipe dream at that point in time.
Service first
Our achilles heel and a constant point of failure were our stats integrations. Only one developer on the team really knew much about it and even then, there were some deep dark recesses that nobody went to. We just prayed they wouldn’t fail. It was a mix and match between XML & JSON feeds, batch uploads and direct calls. Really messy stuff. Even making small improvements for performance could have unforeseen, negative repercussions. A single uncached page could take up to 4 minutes to load while team and player stats were aggregated.
We decided to break all the business logic out into it’s own micro-service. The strategy being that the consumer tech didn’t have to have any logic to handle the data, simply just display it. We focused on the “big six” sports with the first consumer of this being the new SI Mobile app, which we launched January of 2016.
In parallel we needed a way to get at our data. Sharing our data was the key to scalability, with data base transactions being the biggest bottleneck to improving performance. By breaking data retrieval out into a micro-service for sharing our content we achieved 2 things: a standardized way to read our data and limited routes to get it through. This created huge opportunities for us to optimize.
Baby steps
While we worked towards the larger goal, we had to deliver some key projects including migrating MMQB (new JS ad library), The Vault (lots of atomic components), and finally the monster* that is Swimsuit at the beginning of 2016 (componentized approach but far fewer atoms). With each of these projects we tried to figure out how to improve our process, how to structure our CSS, templating languages etc. We were making adjustments to the implementation but also how we worked together with Product and Edit.
Our first crack at the new architecture was SI Kids. We adopted a 3 tier architecture — Back End (Drupal), API Layer (RESTFul JSON), Front End (Node, Handlebars).
We began by creating a white label version of the site, code named Turquoise Unicorn (even has a bad ass logo). The primary focus being that we ended up with a code base we could easily replicate for any of our sites. This site includes all the major functionality we use for articles, section fronts and more. SI Kid provided relatively low traffic environment for us to prove out the model.
Getting the green light
At the beginning of 2016 we started to discuss the reality of a redesign. With the last disastrous redesign still looming in recent memory, I can’t blame folks for being wary of this. Here are these 2 guys saying we need to throw out a sizable investment and start again. After our teams had successfully delivered every project we had embarked on, we had the stakeholders trust. This helped a lot. We could also show through our users feedback, that the valiant efforts of the engineers to keep the site running and performant were not enough.
We would regularly receive feedback like this…
“Dear @SINow: I would love to read @richarddeitsch’s Media Circus. But today, like most days, http://si.com has failed him and me.” - @ProfessorFlynn on Twitter.
and this …
“I love @jonahkeri, but the http://si.com website is the slowest-loading, most terrible website on Earth.” — Chad on Twitter, June 13th, 2016.
While there is a wry sense of humor in these tweets, the issues were real and our site was failing. Our stakeholders agreed this was the right step forward. We needed to do this. The only catch, it needed to be done by the end of July. 2017 right? Ha ha ha, no … 2016. It was February already.
How to start?
The previous redesign had taken a year, roughly 6 months past the original due date, to deliver. We had 5 months. And a much smaller engineering team. We needed to know exactly what we wanted to build and we had to get it right the first time. As Harry Hope once put it, “We are hoping that the system and tools we’ve created can multiply our dev team by 10”.
The temptation would be to run into this head long and start coding. Instead we did the opposite. We needed to define who we were, who our audience was and how we wanted to share our stories with them. We engaged a design agency we had worked with in the past, Moment. They had a process nailed down that would take us from nothing to a full understanding of what we were trying to build with mock ups, in under 8 weeks. It seems like a long time. For a project this size, it isn’t.
We decided to make another big change in that we had all the key people involved from day one. That included leads from Product, Engineering, UI/Design, Edit, Aud Dev, Sales and Marketing.
When we finished this process at the end of March, everyone understood what we were building, why and who for*.
The System & the Process
Traditionally a project of this importance would have required a ton of high fidelity mock ups and designs, with many painstaking iterations and feedback rounds. This would have completely derailed the project and caused massive delays. We needed to make this project as fluid and efficient as possible to hit the deadline.
Instead we focused on getting the stakeholders to understand and approve the system of components and the structure. By doing this we were able to get approval for the overall styling of the system and then focus design on the individual components e.g. scores, player cards, right rail components etc. Once these were reviewed we could begin development immediately.
We were already following an agile methodology on the Engineering team but in order for this to work, we had to get all the teams working the same way. Initially we worked out how many sprints we had between kick off and launch — 8 x two week sprints. For each, we defined some high level goals for completion with the idea that Product and Design were always one sprint ahead of Engineering. This meant that that each components requirements definition and design would be completed and approved so that it could be handed off to Engineering in the subsequent sprint.
After each sprint whatever components had been completed were then reviewed and approved in a live staging environment. This allowed for iterations to happen in real time and everyone from edit and the business were in step with us the entire way. This meant fewer “surprises” and almost no throw away work.
Jump start
Our team is a blend of in office and remote personnel and while we work well together, it was important to make sure everyone was on the same page. We wanted to start the project off with a bang and make a lot of headway very quickly, while setting a good rhythm to the work. Once we were ready, we flew in the entire team for a week. The goal of this week was to come out of it with some base components; primarily scores, schedules and articles.
The first day (Monday) was focused on knowledge share. We went through the materials we had developed with Moment, the mission, the goal, who we were building this site for. We then dived into the system and how we wanted to build the framework. By the end of day 1, everyone on the team, regardless off whether they were actually working on it or not, understood what we were trying to do. The following day we broke the team up into groups — 1 designer, 1 product manager and 1 engineer. Each team would focus on delivering a specific piece of the puzzle by that Thursday.
Throughout the week each group learned how to work together, what each others needs were when it came to requirements, mocks or functionality. It helped the group figure out it’s communication and the end result was so much more than we could have hoped for.
We had working articles and a working sports vertical — Baseball (MLB). Following the principles learned during this intense week, the team would use this base and evolve these processes though out the project.
The end result
I hope the result speaks for itself…
We were able to launch on time on the 28th of July with a site that is clean, performant and extremely flexible. We went from 4 to 5 second average front end load times (excluding ads) to 1 to 2 seconds. On the back end we went from 850–900ms response times to consistent 100–110ms.
Sections that would take a week for a developer to setup, can be setup by editors in 10 minutes.
We’ve created a base that we can easily innovate and build on without worrying what thread we may be unravelling in the moth eaten jersey that was the previous site.
The positive feedback was overwhelming, even getting a tongue in cheek, positive nod from Deadspin.
We’ll keep telling our stories if our readers keep coming back to read them, and when they do, we hope it’s as easy and pleasant an experience as possible. Onward! GOLF.com here we come!
* A story for another time perhaps
