Language Modeling

Wie wäre es, wenn ich als Leser selbst entscheiden kann, wie ich einen Text lesen möchte? Vom Absender zum Empfänger oder vom Lautsprecher zum Kopfhörer so beschreibt Dirk von Gehlen in seinem Buch Meta! Das Ende des Durchschnitts eine Medienkultur, die sich nach dem Empfänger richtet und nicht dem Sender…

As we all know, recently Natural Language Processing (NLP) is getting more attention in the area of AI. How can we define NLP? NLP is part Artificial Intelligence which provide the machines the ability to read, understand and interpret the human languages. There are different real world problems like Sentimental…

Why do we need augmentation? Deep learning revolutionized the field of natural language processing and made a variety of complex tasks much easier to perform. Large annotated datasets play a critical role in this success since deep learning models need a lot of data to be trained on. However, it is sometimes difficult and expensive…

AI2’s open-source QA model now has an interactive demo — Macaw, an open-source question-answering model that we released in October 2021, just received an interactive upgrade. The new Macaw demo allows you to explore and compare Macaw’s answers to those of popular language model GPT-3 over our benchmark challenge dataset of 300 questions. …

What is language modeling? The goal of language modeling is to assign probabilities to sequences of words, such as a phrase or a sentence. But in the simplest case, a language model may also operate at the level of words. If we have a language model, it’s possible to ask questions like what is…

While there are many channels through which customers interact with the brand, voice has been the most preferred channel for customers. Recent advances in audio AI and NLP technologies have placed automated speech recognition (ASR) as the backbone of voice data to make sense of the conversations. ASR is widely…

Como usar uma Rede Neural Recorrente para construir um modelo de língua — Talvez você já tenha visto alguns exemplos pela internet sobre IA' s ou bots treinados em milhares de textos para escrever como um autor específico ou replicar um gênero textual. Diferentes técnicas podem ser usadas para gerar esse tipo de resultado. Hoje, vamos ver como construir um modelo para "escrever"…

Understanding the implicit knowledge “stored” in a Language Model — tl;dr Language models have shown the ability to remember information implicitly from the data used for pre-training. This paper from Roberts et al. (2020) attempts to quantify it and shows how the scale of implicit knowledge varies with model size and training time. Thanks a lot to Sundeep Teki for…

One step forward and the other step backward — Down the memory lane! Not so long ago, Recurrent Neural Networks were the go-to architectures for just about anything that had a sequential nature, most notably for text data. Variants of RNN’s like GRU, LSTM were used for text classification, paraphrasing, language modeling, token classification, and other non-standard problems. However, LSTM models were starting…