Language Model

Recently, Google’s LaMDA (Language Models for Dialog Applications) has been in the news because one of Google’s employees claimed that LaMDA is sentient, and articles that ensued (like this one). When people read sentient, they think human-like and spiral into thinking about human-like behaviours that such an AI possesses. Some…

The next generation language models will still just be enormeous parrot-systems that imitate intelligent behavior. But they will feel like much more to us. — Let’s get this straight out of the way: Current deep learning approaches won’t create conciousness or human-like intelligence. But they quickly become better in imitating it and that alone might cause a lot of problems. Again headlines on new AI as consious beings

We fine-tuned a Large Language Model to summarize Swedish news articles, and the results are impressive! — At Schibsted Futures Lab, we have been looking into how Large Language Models (LLM) can be used to treat text, and how they could be applied within Schibsted, a major media company in Sweden and Norway. LLMs like GPT-3 from OpenAI have the ability to read, interpret and generate text…

This week we feature articles from Lauren Hinkel, and Michihiro Yasunaga, Jure Leskovec, & Percy Liang. Additional reading comes from Antoine Craske, Petr Korab, and Ben Lorica & Kenn So. News comes from Crafter, Siteimprove, MongoDB, and Foxit. Reminder: If you’ve missed any recent issues you can see them here. Opinion / Analysis… LinkBERT: improving language model training with document link …

MobileBERT, Thin Version of BERT, 4.3× Smaller and 5.5× Faster Than BERTBASE — MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices MobileBERT, by Carnegie Mellon University, and Google Brain 2020 ACL, Over 300 Citations (Sik-Ho Tsang @ Medium) Natural Language Processing, NLP, Language Model, BERT MobileBERT is proposed for compressing and accelerating the popular BERT model. MobileBERT is a thin version of BERTLARGE…