NeurIPS 2018 Recap: Large-Scale Machine Learning
Author: Caitrin Armstrong, Research Assistant at Network Dynamics Lab, Machine Learning Engineer at Aifred Health
I went to Las Vegas with my parents when I was twenty. I didn’t want to go at first, thinking that my parents, normally the deep-woods camping types, had gone completely mad. What on earth would I and my teenaged siblings do in Las Vegas with our parents? A lot, it turns out. We had the chance to observe a gigantic spectacle of human ingenuity and abundance. From impressive architecture, over-the-top shows and the nearby Grand Canyon, it felt like everything had been expanded, almost out of recognizable proportion. I needed bigger eyes to take it all in. In the end, it was one of our best family vacations.
Attending NeurIPS 2018 was a very similar experience. Everything was larger than I had expected: the presentation and exposition halls, the workshops, the number of topics being addressed, the parties and the sheer number of people present. I was also a relative outsider: aside from this being my first experience at NeurIPS or even a large conference, I was also someone who had contact only with applied machine learning while the main conference was very theoretical.
So why was I there? Personally, I was eager to soak up an understanding of the state of the field, of the problems being addressed and the state of the art techniques used to solve them. Professionally, my allegiance was split three ways. I was a graduating masters student at McGill University engaged in work in computational social science where I learned and used machine learning techniques as tools for my research. I hoped to perhaps gain insight into new knowledge representation techniques. I have also been, for nearly a year, a machine learning engineer at the Montreal-based precision health startup Aifred Health. As we develop models to aid physicians in better treating depression we face important problems in terms of bias and interpretability; issues I was hoping to hear a lot about at the conference. Finally, on the virtue of both of these positions, I had also been selected as the Montreal ambassador for NextAI which gave me the chance to speak with many early-stage startups eager for funding and support.
Intellectually, the conference itself was intense. The schedule was also intense! It was so overfilled that I and Aifred CTO, Robert Fratila set aside several hours the week before to read through abstracts to schedule our plan of attack. It was a tough challenge to identify which of the talks, spotlights, poster sessions and workshops would be of both interest and value. I had to guess which I had enough of a background to understand and which would be completely over my head. I didn’t make it to all of the talks I identified, but taking that time to plan was definitely a good move, as I had a pared-down list of options ready to go during the hectic conference.
Others have summarized the actual content of the conference, so I’ll only mention a few personal highlights. Joelle Pineau gave an invited talk to a huge audience revealing the distressing lack of statistical inference tests in the machine learning literature, likely upping the standards of the entire field in the process. During the spotlight talks I got a whirlwind introduction to the newest advances in transfer learning and machine translation.
In general, I found I got the most value out of the workshops. They were slower paced and more focused. Given my focus on applied machine learning, I was the most drawn to the workshop on Critiquing and Correcting Trends in Machine Learning and the ML for Health workshop. The correcting and critiquing ML workshop felt much more intimate, and a breath of fresh air compared to the intensity of the hype elsewhere. I appreciated the reminders, as a young researcher and practitioner, to critically examine the impacts of my model choices. In this and the ML for Health workshop I learnt more about counterfactuals and domain adaptation, both important for ML application for sensitive real-world data. The AI for Social Good workshop reminded me, surprisingly, of the role of unions in my future career, something that I as a young 21st century worker had nearly forgotten about.
Emotionally, the conference was inspiring and exhausting. As a grad student, I’m not used to interacting with dozens of people on a daily basis: the conference was much more socially demanding than I expected! As the week went on I got better and better at introducing myself succinctly, gaining little tidbits of information and connecting with every person I talked to. There were many extravagant company parties in the evenings, most of which seemed to be invite-only. Personally, I had surprisingly little trouble getting invitations, I suspect this is because I was a young female engineer from a relatively well-known institution. Although I worry a little about the inclusiveness of the invite-only model, there were also plenty of other avenues for socializing. The Whova conference app allowed users to create topics of interest and plan social get-togethers, a welcome tool given to the 8000 participants. The amount of money and enthusiasm at the conference was insane.
On the first day of the conference, I attended the Women in AI workshop, switching in between the WiML sessions and the regular conference sessions. A number of the women there were from machine-learning adjacent fields: business women or graduate students eager to understand machine learning and support their fellow women in tech. Joelle Pineau gave a speech at the welcome dinner, addressing the silent question some of us were asking: “what is the role of women in ML today, given as we fill an entire banquet hall?”. She reflected on how the early women in machine learning workshop felt like “a question of survival”, but also on how the still-present gender disparity and controversy over the name change meant that there was still a need for a place for women in machine learning to come together for support and celebration.
Overall, the conference was an eye-opening event on a huge scale. I went expecting to largely learn about the newest and the greatest in machine learning, but came out with social connections and a stronger conviction in regards to the need for critical thought in the implementation of machine learning, despite its enormous potential. I’m extremely grateful to NextAI for the opportunity!
