Sotawhat, Dynamic Meta-Embeddings, Journal, Fairness in ML Course, GraphNets, NLP Overview Paper, Medical Torch,…
Topics in this newsletter include latest advancement in deep reinforcement learning; history of neural network architectures; Facebook’s dynamic meta-embeddings,…
Hi and welcome to the 31st issue of the NLP Newsletter! I am Elvis and here is a recap of this month’s notable NLP news: latest advancement in deep reinforcement learning; history of neural network architectures; Facebook’s dynamic meta-embeddings; lots of machine learning and deep learning educational material; latest news on ethics in AI and AI for social good, and much more.
On People…
An MIT research team are developing an AI algorithm that can automatically create pizza recipes and make graffiti. The idea with this research, besides being fun, is to help people develop deep intuitions about machine learning’s capabilities and to help us in our creative endeavors — Link
Learn a bit about how AI companies want to embed common sense into machine learning algorithms to be able to improve performance on certain critical tasks such as driving autonomous cars and performing medical diagnoses — Link
Here is a 60-minute self-study training module on fairness released by Google as part of an initiative to help students get up to date on topics related to ethics in AI and fairness in machine learning — Link
Listen to Bushra Anjum, Technical Lead at Amazon, discuss the importance of having an interdisciplinary mindset and focusing on core human values, such as creativity and empathy, when building technologies — Link
MIT announces $1 billion initiative to help teach students and AI researchers about the social and ethical implications of AI — Link
On Education and Research…
Deep reinforcement learning is an important area of research today. Take a look at Joyce Xu survey on the latest trends in advanced reinforcement learning. She covers topics that go beyond DQN/A3C and introduces a class of reinforcement learning known as hierarchical reinforcement learning. In addition, the survey discusses other important areas of research, such as memory and attention, and how they can be incorporated into the reinforcement paradigm of machine learning — Link
Afshine Amidi releases a website that contains machine learning cheat-sheets accessible in various languages such as Spanish, Italian, Arabic, and French, among others— Link
Do you often get frustrated with getting your latest batch of research papers and SOTA results that are relevant to the topics you are interested in studying? If you are, Chip Huyen built a handy little crawler to help you efficiently query for the latest abstracts and summaries from arXiv — Link
Young and colleagues have released an update of their recent paper on “Recent Trends in Deep Learning Based Natural Language Processing.” Their revised version includes the addition of SQuAD state of the art results as reported in the current NLP literature — Link
Eugenio Culurciello, in a recent post published earlier this month, provides a nice history of neural network architectures and what motivated the design of each one (see chart below for an overview of architectures)— Link
Various machine learning research has been happening in the health space:
- Deep learning algorithms were used to detect critical findings in head CT scans — Link
- Google has developed an algorithm to detect the spread of breast cancer — Link
On Code and Data…
Fast.ai partners with AWS Open Datasets to standardize, host, and release open datasets on an infrastructure with high reliability — Link
If you want to review your math using a tool like PyTorch and Numpy, fast.ai has built a set of tutorial videos where Rachel Thomas and Jeremy Howard teach the fundamentals of computational linear algebra. Topics include implementation of topic modeling algorithms, PCA, linear regression, among others— Link
Words embeddings are mostly used for text classification and solving other conventional NLP tasks such as sentiment analysis. They could also be used for far more sophisticated systems such as building a collaborative recommendation system as reported in a recent blog post by Sylvain Truong (see figure for a preview of the framework) — Link
DeepMind releases GraphNets, which is an easy-to-use library for training graph-based neural networks — Link
deeplizard releases a new course on the fundamentals of neural networks and tensor math, taught purely with PyTorch — Link
Learn how MIT researchers are building an infrastructure based on AI technologies to crowd-source data, which they use to build a system that can predict new drug-protein interactions — Link
dair.ai published a new tutorial which teaches how to perform deep learning based emotion recognition (known as a complex NLP task) using both PyTorch and TensorFlow deep learning frameworks — Link
A new medical imaging framework (Medical Torch), based on PyTorch, has been open sourced by Christian Perone. The idea with this tool is to simplify the steps of pre-processing MRI data and other images used to analyze internal parts of the human body like spinal cord gray matter — Link
On Industry…
Facebook AI research team releases a new paper highlighting key findings on a method that dynamically selects the right types of embeddings for a specific task at hand. They call this concept dynamic meta-embeddings and it tends to outperform traditional word embedding on a variety of tasks — Link
On the topic of NLP, so far this year we have witnessed many breakthroughs in the field of natural language processing, starting from models that learn deep contextualized features (ELMo) to models that leverage fine-tuned language modeling techniques to efficiently perform downstream tasks such as sentiment analysis (ULMFit). More recently, a new system has been proposed by Google AI Research team, called BERT, which basically tweaks the way contextualized featured are learned in the Bi-LSTM layers of a language representation model — Link
Journal is an ML and text mining platform that allows you to make searches across all your existing web services and tools. Journal, which is backed by Social Capital, has been released in beta mode and is intended to be used as a tool to help you find information faster and more efficiently — Link
Wall Street firms and other big start-ups are now investing heavily on applying NLP and machine learning to help consumers manage their money and plan for retirement (a sort of automated financial manager if you will) — Link
Worthy Mentions…
Take a look at our new series called “AI Diary” where we discuss some of the latest trends related to machine learning and natural language processing, including special topics such as ethics in AI, AI for social good, the future of AI, among other interesting topics. Besides news, it also covers commentary about pressing issues in machine learning research and AI technologies — Link
Check out this useful newsletter (The Creative AI newsletter) that provides updates (resource, opportunities, etc.) on everything related to AI art, music, and design — Link
This basic tutorial teaches you how to conduct medical imaging analysis using PyTorch and Python. It also provides a code walk-through on how to use CNNs to train a spinal cord gray matter segmentation model — Link
The following e-book describes the key math concepts used in classical machine learning — Link
One last quick thing: any sort of engagement (like follows, shares, 👏👏👏, and feedback) will make a huge difference for the future and sustainability of the dair.ai publication. So I will deeply appreciate any of that in advance.
