Building AI Superclusters in Canada
The Canadian government is determined to build AI ‘superclusters’ in Toronto-Waterloo, Montréal, and Edmonton to utilize the country’s top research resources.
This is the first article within featured series AI and Globalization, click to read this story in Chinese, or Canvassing Switzerland’s AI Landscape.
Like Wall Street for finance, Silicon Valley is the American centre for tech research and development — the place where it all comes together. Up in Canada meanwhile the scale is much smaller, and the country has not yet fostered a tech hub remotely comparable in size to Silicon Valley. But that may be changing, thanks to a number of powerful stakeholders’ determination to build AI superclusters in Toronto-Waterloo, Montréal, and Edmonton.
AI in Canada: Hinton and CIFAR
In 1987, Dr. Geoffrey Hinton quietly moved north to Canada, accepting a tenured position at the University of Toronto. He claimed to want to avoid funding from the US military research program DARPA, which had been supporting AI research for decades.
Upon his arrival, Hinton joined the Canadian Institute for Advanced Research (CIFAR) and their first research program Artificial Intelligence, Robotics & Society.
The neural network method he focused on was revolutionary, seeking to make machines learn by emulating how neurons function in the brain. Yet it didn’t really work at first because computation was time-consuming and processors weren’t good enough to support it. In 1986, Hinton, alongside David Rumelhart and Ronald J. Williams, had demonstrated that backpropagation could optimize neural networks, but datasets were too small to train these models and application scenarios were limited. Researchers used models with fewer layers such as support vector machines (SVM), or boosting instead.
In 2004, Hinton persuaded CIFAR to fund another project, Neural Computation & Adaptive Perception. This kickstarted the next round of AI fervour, putting neural network research in the public spotlight, and garnering further funding.
Serious Scientists Don’t Give Timelines; Yet Business Runs in Real-Time.
The breakthrough came in 2006: Hinton led a published paper called A Fast Learning Algorithm for Deep Belief Nets, which first proposed the method of greedy layer-wise training for deep neural networks. In an competition run by ImageNet in 2012, Hinton’s UofT team used convolutional neural networks (CNN) for image recognition application. Given the large pool of image datasets and the computation power of GPU processors, the team was on the right track, and their results redefined the field of computer vision.
Two of Hinton’s earliest correspondents were Yoshua Bengio from the University of Montreal and his own postdoc student Yan Lecun, who joined Hinton’s UofT lab in 1987 and now leads AI research at Facebook. The accomplished trio is sometimes jokingly referred to as the “Canadian Mafia” of deep learning.
The three researchers still had to wait, however, for data and hardware to catch up with their ideas. This required patience, and although Bengio quipped at a Toronto machine learning event that “serious scientists don’t give a timeline”, the business world nonetheless felt tectonic shifts in real-time: machine learning has immediate applications in image recognition, voice recognition, translation, sentiment analysis and much more. The technology is already here, and as it hits the market, everyone is affected.
Building AI Superclusters: Toronto-Waterloo, Montréal and Edmonton.
As we’ve seen in Silicon Valley, geography matters when it comes to innovation. Resources amass around the most important city nodes. In Canada, Toronto-Waterloo, Montréal, and Edmonton are the new AI superclusters that funding, business and people are gravitating toward. These three locations topped the federal nomination list in the 2017 Canadian federal budget. Now they are officially on the national VIP list for artificial intelligence research and development.
Toronto-Waterloo Chapter
Downtown Toronto’s 140,000 square-metre MaRS Discovery District features an Edwardian-style Heritage Building, linked by a spacious glass-roofed Atrium with a new South Tower and the Toronto Medical Discovery Tower.
Synced spoke with CIFAR’s government relations officer Brent Barron at his MaRS office — an open-concept space overlooking the Ontario Legislative Building. “At the moment, CIFAR is doing three things with AI: one, working with researchers including Geoffrey Hinton on the Learning in Machines & Brains program, an interdisciplinary research program where AI meets neuroscience; two, working with the three AI institutes; and three, investigating the economic and social impacts of AI.” In addition, CIFAR operates a summer school for AI training.
MaRS is home to the Vector Institute, a ten-minute walking distance from the University of Toronto, Creative Destructive Lab, and Hinton’s office at UofT. Within twenty-minutes walking distance is Google Brain Canada and NEXT AI, the newly founded AI startup incubator. The neighbourhood is also convenient to Toronto City Hall, the financial district, and shopping centres.
We are looped into an AI ecosystem by default — with education, money, incubation, and industry connections all closely mapped. The district is further supported by the federal and Ontario provincial government’s C$100 million investment (plus an additional C$80 million from private industry) in the Vector Institute.
“The Vector Institute will confirm Canada’s world-leading position in the field of deep learning artificial intelligence,” said TD Bank CEO and Chair of the Vector Institute Board of Directors Ed Clark at the opening ceremony in March. “It will spur economic growth in Canada by attracting talent and investment, supporting scale-up firms and enabling established firms to be best-in-class adopters of artificial intelligence.”
Toronto’s aim is clear: it was the birthplace of paradigm-shifting research, it has money and spacious new facilities, and now it’s aiming to attract and retain top talents in the field.
If we hop into a car and drive for less than two hours — approximately 100 kilometres — we reach the city of Waterloo. During the 1990s, Li Deng, former professor at University of Waterloo, led a group of 20 graduate students and several postdoc fellows in AI research here. Deng says his Waterloo group “was focusing on AI applications to speech recognition, natural language understanding, and signal processing, where neural networks were shown promising but not competitive with other machine learning methods during those older days.” Their AI and speech research from 1989–1999 was funded by Nortel Canada, Bell Canada, the Natural Sciences and Engineering Research Council of Canada (NSERC), the Ontario government, the Department of National Defense (Canada), and the National Science Foundation (NSF of US), among others.
Dr. Deng left Canada at the end of 1999 to join Microsoft Research in Seattle/Redmond. Deng and Hinton co-organized the NIPS workshop on Deep Learning for Speech Recognition in 2009. “Our connection during 1990s in the Toronto-Waterloo area helped me bring Geoff [Hinton] to Microsoft in Redmond to consult for me in 2009 and 2010, after we saw the time was right to revive neural nets,” says Deng.
The “impact factor” of the Toronto-Waterloo region is weighted into a global network, even if researchers leave the country they continue to be bonded by common research projects.
Edmonton Chapter
The Alberta Machine Intelligence Institute (AMII), Reinforcement Learning and Artificial Intelligence Group (RLAI) and Bionic Limbs for Improved Natural Control (BLINC) are all located on the campus of University of Alberta. Of the three, the AMII has the most researchers — up to 100 — while some of its crew overlap with the other two labs. It conducts research in reinforcement learning, deep learning, statistical learning, natural language processing, and social network analysis.
Dr. Richard Sutton is involved with all three labs. He is considered to be the father of reinforcement learning — a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives while interacting with a complex, uncertain environment. It mirrors the concepts of Pavlov’s Dog and classical conditioning theory in psychology. Nowadays, researchers are combining deep learning with reinforcement learning to create deep reinforcement learning (DRL). In fact, the recent success of AlphaGo can be traced to a powerful combination of the Monte Carlo tree search and DRL techniques.
Cameron Schuler, the Executive Director of AMII, says, “[Canada] has a huge lead in research. Hinton, Sutton and Bengio are all great thinkers. By area, Toronto is predominantly deep learning, Montreal is deep learning and reinforcement learning, whereas Edmonton has a significant focus on RL — but our researchers have diverse interests and broadly apply all aspects of ML & AI.”
Alberta’s economy is largely built on oil & natural gas, agriculture and forestry — which are also vital Canadian industries. Machine intelligence has the capacity to upgrade some of the energy industries, reducing fixed-costs and optimizing the process of extraction based on currently available datasets. According to the management consulting firm McKinsey & Company, “adoption of these technologies [in the oil & gas industry] could unlock cost savings of between $900 billion and $1.6 trillion in 2035, equivalent to the GDP of Indonesia.”
It is predicted that the global consumption of energy will peak in 2035, while we pivot towards cleaner energy. It is unlikely that AI will be the new “it” for Canadian industry right away, but it will definitely transform important business sectors in Edmonton. AI Startups like OneBridge Solutions are utilizing data analytics to revolutionize pipeline management. The company also partnered with Microsoft and has access to Microsoft’s software service.
The Province of Alberta has provided AMII with C$40 million in funding over the last 15 years. Schuler says the most important thing is to build up a local ecosystem, but this is neither quick nor easy. Regarding the number of startups, Alberta is falling short. Edmonton’s startup landscape is about 1/6 the size of Toronto’s or Montreal’s — which together are incubating over AI 100 startups.
Retaining AI talent in Alberta is an issue, and Schuler recognizes the importance of working with a group of closely-knit researchers. “My group is the only group I know that didn’t lose anyone to industry,” he says. “Our professors chose this path — in Edmonton, you work with people you know.”
Montréal Chapter
“You don’t define the world through nations, you see the world through cities,” proclaimed the Mayor of Montréal at the opening of the recent C2 event, where local startup Element AI partnered to host the 2017 Artificial Intelligence Forum, part of a vibrant fusion with arts, design and industry participants. Montréal is traditionally regarded as a cultural hub, and now the city wants to add AI to its profile.
The federal government has given Montréal Universities special research grants totalling C$213 million, while the provincial government also plans to invest C$100 million over the next five years.
There are currently 15 startup incubators in the city. Element AI Founder Jean-François Gagné recalls that 15 years ago he felt like hiding after telling people he worked in the AI industry. Today, Gagné partnered with renowned AI researcher Yoshua Bengio and ambitiously wants to help make every company in the world “AI first.”
According to Element AI, the company “is building a universal AI software platform that helps resolve business tools, bringing business on board in a long-run development.” This includes augmenting businesses with capabilities like natural language processing, voice recognition, predictive analytics, targeted-advertising, and much more.
After a modest start with an office in historic Notman House, the company is now expanding its office space to accommodate an estimated 400 people by the end of this year. Element AI is aiming to become an international “bridge” for global AI open access by offering strong AI expertise. “We are P-to-P [paper to product], fostering research into functional applications in-house. We provide a complete feedback loop, integrating researchers, programmers, product managers and business,” says Alex Shee, programmer director at Element AI. One month after our visit, the company raised C$135 million in series A funding, a historic high for AI companies.
While Element AI helps build relationships with businesses, Bengio also heads the MontrealInstitute for Learning Algorithms (MILA) on the campus of Université de Montréal. MILA is supported by the Institute for Data Valorisation (IVADO), a closely-knit group formed by four academic institutions: École Polytechnique de Montréal, Université de Montréal, HEC Montréal, and McGill University. IVADO received $93 million in standalone funding in 2016.
Why and Why Not Canada: Data, Money and Globalization.
When asked to contrast Canada with the US for AI research, UofT’s David Duvenaud, who has conducted research in the US and Europe, says, “If we prefer the US, it’s because of higher salary, higher international profile, better weather, and easier access to very large (but hard to get) grants. If we prefer Canada, it’s because of the easier access to medium-sized federal and provincial grants, lower personal costs, and better political climate and immigration rules.”
One of the biggest concerns for Canadian startups is the ability to scale. 500 Startup tells us that their investment strategy is to focus on hatching small and medium sized businesses. Their VC have invested in over 30 companies in Canada, with co-investors targeting startups with early market validations, growth revenue and real products. Ideas that require big infrastructure support are less likely to be realized in Canada. Currently the most popular AI application startups in Canada are business analytics, cyber security, fin-tech including personal finance, social and real estate.
Canada has struggled to foster big unicorns, and its startups tend to sell after reaching medium size. The most recent scenario is Maluuba, acquired by Microsoft this year. This is a way to tap into the resources big corps have to offer, the downside being the startups often proceed under American ownership.
AI research is powered by people, funding and data. To train the next algorithm model, one wants the biggest datasets and the best people in the field. Google, Facebook, Amazon, Baidu, Tencent have them, while Canada has trailed behind.
However, there are ways to work around this according to Yoshua Bengio, “for datasets, we can always make deals with big corporations. Also a lot of the research we do now utilizes public domain data, which is plentiful. Canada also has its own niche for social healthcare data.”
Intelligent machines are also capable of creating their own datasets under certain application scenarios. Cepheus, the research product of Dr. Michael Bowling’s team at University of Alberta, is a program that plays heads-up limit “Texas Hold ’em” poker, and perfectly. The program was given no human expert help or external datasets. It was only granted the rules of the game and trained against itself, by playing billions of hands of poker.
As for funding, big tech’s resources and R&D centres are being increasingly outsourced to the north. Last year, Google invested C$4.5 million in the MILA and C$5 million in Toronto’s Vector Institute; Microsoft has promised C$6 million to the Université de Montréal, C$1 million to McGill University, and doubled its investment in AI research in Toronto; Uber recently expanded its Advanced Technologies Group (ATG) to Toronto under the supervision of Raquel Urtasun from UofT; and 500 Startups Canada established a C$30 million fund to invest in Canadian startups. Google’s parent company, Alphabet, is also applying to the city of Toronto to develop a high-tech district in downtown Toronto named “From the Internet Up.” The application is the latest initiative from Sidewalk Labs LLC, the company’s urban innovation unit, and is part of a vision to create a large-scale urban district modeled after a tech company.
Many Canadian industry insiders question whether big tech outsourcing their R&D to Canada is a good thing, but the trend seems to follow the natural flow of globalization. The 800 Canadian PhD students graduating in the coming years are more likely to stay. The Canadian superclusters are being built, even with American big tech companies on board.
The federal government is backing everything AI with the Pan-Canadian Artificial Intelligence Strategy, which is pumping C$125 million into R&D. Overall, the nation has attracted C$1 billion in AI funding.
While the Canadian total is humble compared to Elon Musk’s US $1 billion investment to OpenAI alone, it is a good first step, considering that the market for artificial intelligence-related products is predicted to reach $127 billion in 2025, while global venture capital funding has grown to $5 billion in 2016 [1]. The market is huge and contains multifarious applications. While the US government is cutting R&D funding, the Canadian government is not taking a passive approach to AI development.
Canada is known for its steady political climate, which is welcoming to international talents and money. Companies that lack the revenue streams of Google can still hire top-notch North American talents with global backgrounds in Toronto, Montreal or Edmonton, without worrying whether they might be asked to leave the country. This creates a great international profile that is attractive for Asian, European, or even American investors and entrepreneurs.
While scientific paradigm shifts can put researchers off balance, it is very possible that the next big commercial idea can emerge in Canada, just like it did not long ago, clandestinely in research. And so, it is never a bad idea to be prepared and get on board.
Canada’s new superclusters are cooperating to achieve this common goal. “We are taking active steps to this. As a country of 35 million people, we can’t compete against huge ecosystems if we are not together in the run,” says Element AI.
“If you do AI research in Canada, it’s got to have a Canadian feel to it — reflecting the climate, the people and their beliefs. We are inclusive and diverse and this extends into the research we do and the organizations that back it. We do a lot of investment in clean tech and solar energy, and also, let’s say we collect cultural heritage datasets, we would include Aboriginal input, since a lot of their information remains invisible now,” says CIFAR’s Barron. Asked if he’s optimistic about the future, he replied: “I would say yes!”
We can expect the new Canadian superclusters to boost international open access and development in AI. Ultimately, the development of AI is crucial not only to interdisciplinary research and business applications, but also to society at large. Recalling Hinton’s journey up north two decades ago, we can ask ourselves: Why not Canada?
“It’s cold in Canada” was the biggest gripe we heard. With AI heating up the way it is, climate is hardly a deal-breaker.
“The 2016 AI recap: Startups see record high in deals and funding.” CB Insights blog, January 19, 2017, https://www.cbinsights.com/blog/artificial-intelligence-startup-funding/.
Journalist: Meghan Han | Editor: Michael Sarazen | Producer: Chain Zhang
