By Jonathan Frankle

Photo by Peter Beukema on Unsplash

Neural networks — the technique underlying nearly all systems we describe as “artificial intelligence” today — are revolutionizing the way we communicate, get around, and express ourselves. In one of the most prominent recent examples, the GPT-3 model from OpenAI has learned to automatically generate web page layouts, search queries, and entire blog posts.

Although these new applications are unquestionably extraordinary, the neural networks that make them possible have become enormous. Consider GPT-3: it needs to run data through 175 billion individual parameters every time the model is used. These parameter-counts have been increasing dramatically in recent…


This post originally appeared on the IBM Research blog.

At IBM we’re committed to an exploratory science agenda, working with companies to advance innovation research and learning within their ecosystem. Today, IBM is embarking on a multi-year, collaborative effort with Wells Fargo focused on research and learning that is intended to enhance the company’s artificial intelligence and quantum computing capabilities. Together with IBM, Wells Fargo plans to accelerate its learnings to inform innovation initiatives that reimagine the future of financial services in a way that is designed to deliver customer experiences that are simple, fast, safe and convenient.

As part…


Rapid advancements in the field of artificial intelligence (AI) are uniquely poised to transform entire occupations and industries, changing the way work will be done in the future. The advent of AI will very likely shift the demand for labor skills. It is imperative to understand the extent and nature of the changes so that we can prepare today for the jobs of tomorrow.

New empirical work from the MIT-IBM Watson AI Lab uncovers how jobs will transform as AI and new technologies continue to scale across business and industries. We created a novel dataset using machine learning techniques on…


While there is a growing body of work studying the theoretical properties of neural networks, our understanding of the macroscopic behavior of deep learning still leaves much to be desired. Questions persist around what drives the evolution of internal representations during training, the properties of learned representations, and how fully trained networks process information. In addition, much of what we do know is informed by anecdotal results.

The Information Bottleneck theory attempts to address these questions. Working closely with collaborators from MIT as part of the MIT-IBM Watson AI Lab, our ICML 2019 paper, “Estimating Information Flow in Deep Neural…


Many human activities require collaboration, including soccer, medical treatment and performing arts. The high level goal of our research is to develop intelligent automated agents that can perform these tasks as well as humans. These tasks in general can be formulated as multiagent reinforcement learning (MARL) problems that involve sequential decision making. In cooperative settings, where agents interact to work together as a team in an environment, the agents receive local observations and a shared team reward at every time step. Without knowing the underlying dynamics of the environment, the goal of MARL is to learn a behavior policy for…


Figure 1: An overview of robustness evaluation algorithms and robustness certification algorithms for neural networks.

Introduction

In this post, we briefly review a recent line of research on evaluating the robustness of neural network and its research progress, with a focus on the joint work accomplished by the MIT-IBM team. We start by giving an overview of this research stream. We then introduce three of our contributions along this line: the first robustness estimation score CLEVER, and two robustness lower bound certification algorithms, CROWN for neural networks with general activations and CNN-Cert for general convolutional neural network (CNN) architectures.

CNN-Cert: An Efficient Framework for Certifying Robustness of Convolutional Neural Networks

Overview

Assuming that attacks are Lp norm…


Teaching AI to generalize from one domain to another would allow it to perform better and learn faster, which would be a huge leap forward for advancing AI. Deep neural networks, trained with a large amount of labeled data, can fail to generalize well when tested with examples from a target domain whose distribution differs from the training data distribution, referred as the source domain. It can be expensive or even infeasible to obtain the required amount of labeled data in all possible domains. For example, say you have sports videos where you train a system to recognize the actions…


Drug discovery is a long and tedious process; it may take years of development and exploration by expert chemists and pharmacologists. In drug discovery, de novo design (i.e., designing an entirely new molecule from scratch) plays a crucial role. Aiming at bringing the power of AI to chemical research, we introduce a new machine learning approach to graph generation and successfully apply it to automate the molecule design process.

The recent years have witnessed rapid progress in the development of machine learning techniques for the generation of a wide variety of data, including images and sequences. There remain, however, substantial…


749 attendees, 85 companies, 27 universities, 17 events — all packed into 1 amazing week of celebration, workshops, open collaboration, and fun. This was the scene at the inaugural AI Research Week, hosted by the MIT-IBM Watson AI Lab, in Cambridge, MA, October 1–5, 2018. AI Research Week was created to bring together the top minds in AI to formulate research directions, network, and share their successes in topics that are crucial to the advancement of AI.

Dario Gil (IBM VP of AI and Quantum) and Anantha Chandrakasan (MIT Dean of Engineering) kick off the AI Horizons Colloquium

The opening events of AI Research Week were the AI Horizons Network (AIHN) Annual Meetup and Colloquium. During this event, AIHN collaborators came…


IBM Accessibility Research has recently been exploring the topic of fair treatment for people with disabilities in AI systems. Such systems must be carefully designed to avoid discrimination against marginalized groups. Are people with disabilities at risk of being disenfranchised by AI systems, and how can we change this?

Attendees at the ‘AI Fairness for People with Disabilities’ workshop

To explore these questions, IBM Accessibility Research hosted a workshop on ‘AI Fairness for People With Disabilities’ earlier this month, as part of AI Research Week hosted by the MIT-IBM Watson AI Lab. Our workshop convened a diverse group of people with disabilities, representatives of advocacy organizations, AI specialists, and accessibility…

MIT-IBM Watson AI Lab

This is the official Medium account of the MIT-IBM Watson AI Lab. The account follows the IBM Social Computing Guidelines. @MITIBMLab

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store