CBC.ca: Where We’ve Been, Where We’re Going
Today, CBC/Radio-Canada’s suite of sites encompasses millions of pages, averaging more than 17 million Canadian visitors every month. CBC.ca’s content publishers produce hundreds of articles a day and we make that content available on desktop and mobile, and on five core apps with a presence on six different platforms. We serve text, images, audio, and video that represent content from News, Sports, Television, Radio, Arts, Comedy, Music and more. The technology we use to do so has evolved dramatically since we launched and continues to evolve. We are going to be using this blog to give you a look at how it all works.
Where we’ve been
Twenty years ago, CBC.ca got its start as a conglomeration of <table>-based HTML pages running off a handful of Sun servers. The server that hosted the CBC TV website actually doubled as our DNS server. As we scaled up, we built a Java and Perl based content management system to dynamically deliver site content. This system eventually fell prey to our own success. Despite serving over two million visitors a month, the site was brought down when over a million people visited in one night to see live coverage of the Quebec provincial election in 2003. Having to anticipate massive traffic spikes forced us to reimagine how the site was built.
In late 2005, we started trading dynamic site generation for a customized version of TeamSite that pre-generated flat HTML files to ease the load on our infrastructure. Subsequent improvements in infrastructure, caching, and content delivery networks allowed us to scale from three million visitors a month in 2006 to six million a month in 2012. We were also able to start revisiting dynamic, database-driven content management tools.
In 2013, we switched our content publishing activities to two new CMSes: Polopoly and Expression Engine. Polopoly is a Java-based platform that we customized to make publishing articles as easy as possible. Expression Engine gave us a more configurable tool we could use to build custom micro-sites and manage content that didn’t fit easily into the template of an article. These two systems, sitting behind the Akamai CDN and a Varnish caching layer, make up the majority of the content you now see on our site.
This arrangement has served us well, but we know it won’t work forever. We have already started to think about what comes next.
Where we’re going
The web has always been a place that changes quickly. In the past, we’ve been able to periodically upgrade our tools and technology to keep pace with our growth. These days, that isn’t enough. We can’t just upgrade, we have to figure out how to upgrade faster.
Because of this, the work we’re doing now isn’t just about changing the technology we’ve got, but also about making it easier to iterate on our products and change that technology again in the future. The implications of this are far-reaching.
It starts with making sure we can test and deploy code as quickly as possible. We’ve already incorporated Bamboo as a continuous integration and deployment tool, but are still trying to improve our processes to take full advantage of what it can offer. Part of that is a focus on increasing automated test coverage, and so we’ve been hiring automated testing specialists to help us upgrade our testing procedures.
Where we once required dedicated hardware or virtualized environments to host our applications, we are now starting to explore how containers can be used to make development more efficient. Building and deploying smaller applications in containers gives us more freedom to choose where we host (on our own servers or in the cloud) and what technology we use to build.
Historically, we’ve relied on monolithic content management systems to produce, organize, and present content. We’re now shifting toward APIs so that our content can get everywhere it needs to go more efficiently. Each step in the process can be optimized independently if we build effective interfaces between them. Our content is still being produced using the same systems, but we are leveraging APIs on top of them so that we can organize our content in different ways and present it natively wherever it needs to appear.
Even the underlying protocol of the web is changing. HTTP is giving way to HTTPS which will give way to HTTP/2. We’ve already begun the process of switching our site to HTTPS delivery so we can provide security for our audience and take advantage of new browser features like service workers. Soon after, we’ll be able to get all the performance benefits of HTTP/2.
Although we’ve already experimented with some of these changes, they all come together in the new version of the site we started rolling out gradually last fall. It is built using containers, is deployed to the cloud, runs on Node.js as a universal React application, gets content from our APIs, and will be secured with HTTPS.
This blog will give us a chance to provide some detail about the work we’re doing, the challenges we’re facing, and the solutions we’re devising. We’ve already started work on articles that will dive into some of these topics, so if you’d like to hear more about how CBC.ca works, be sure to give us a follow!
