kAi Weekly Newsletter — July 27th, 2023
Writer: Fatih Sarıoğlu, Kutluhan Aygüzel and Kourosh Sharifi
Welcome to our weekly AI News Digest! Here, we’ll bring you the latest updates on Artificial Intelligence. Discover the most exciting events happening in the field of AI, their impact on the world, and what’s to come in the near future. Stay tuned for more exciting insights and breakthroughs by following our newsletter!
Stability AI Releases Stable Diffusion XL 1.0
In this week’s AI news, Stability AI, an AI startup, has introduced its latest release, Stable Diffusion XL 1.0, a text-to-image model that boasts more vibrant colors, better contrast, and improved text generation. With 3.5 billion parameters, the model can produce high-resolution images in seconds. Stability AI has taken steps to address ethical concerns by filtering out unsafe imagery, but the open-source nature of the model raises the potential for misuse in generating harmful content and deepfakes.
While the new model presents exciting advancements, it also faces challenges related to copyright infringement, as some artists have protested against the use of their work as training data. Stability AI has partnered with another startup to respect opt-out requests from these artists. Additionally, the company is working on improving safety functionality and is committed to incorporating artists’ requests to be removed from training datasets.
To enhance user customization, Stability AI is releasing a fine-tuning feature in beta for its API, allowing users to specialize image generation for specific people and products with as few as five images. The company is also collaborating with Amazon Web Services (AWS) and bringing Stable Diffusion XL 1.0 to Bedrock, AWS’s cloud platform for hosting generative AI models.
Despite these innovative moves, Stability AI faces stiff competition from other players like OpenAI and Midjourney. The company recently raised significant venture capital but reported a financial lull, leading to a $25 million convertible note and an executive search to boost sales efforts.
Stability AI’s CEO, Emad Mostaque, expressed enthusiasm for their latest release, emphasizing the company’s commitment to collaboration with AWS and providing cutting-edge AI solutions to the community.
Source: TechCrunch
OpenAI Quietly Shuts Down Its AI Detection Tool
OpenAI, the leading artificial intelligence powerhouse, recently announced the discontinuation of its AI detection tool, AI Classifier, which was designed to detect content generated using AI models like ChatGPT. The tool was withdrawn due to its low accuracy rate, as it struggled to distinguish between human-written and AI-generated text effectively.
Initially introduced as a potential solution to identify AI-created content and prevent misuse, the AI Classifier was labeled as “not fully reliable” by OpenAI itself. Evaluations revealed that it correctly identified only 26% of AI-written text as likely AI-generated, while also mistakenly labeling human-written content as AI-generated in 9% of cases. The limitations of the tool included unreliability with texts under 1,000 characters and poor performance of neural network-based classifiers outside their training data.
Education, in particular, expressed keen interest in AI detection to address concerns of students using AI chatbots like ChatGPT to write essays. However, OpenAI acknowledged the importance of recognizing the limitations and impacts of AI-generated text classifiers in the classroom. The unreliable nature of the AI Classifier raised concerns among educators.
OpenAI is now focused on incorporating feedback and exploring more effective techniques for text provenance. The company aims to develop mechanisms that allow users to identify whether audio or visual content is AI-generated. As the usage of sophisticated AI tools becomes more prevalent, the demand for reliable AI detectors has risen.
The discontinuation of the AI Classifier highlights the challenges in accurately identifying AI-generated content and underscores the need for further research and development in this area. OpenAI remains committed to refining its AI tools and plans to expand its outreach efforts to gather more insights from users. While the AI Classifier did not meet expectations, OpenAI’s ongoing efforts indicate a commitment to address concerns surrounding AI content and its impact on various domains, including education.
Source: decrypt.co
Worldcoin: Sam Altman Launches His New Eyeball Scanning Crypto Coin Project
AI entrepreneur Sam Altman has launched a controversial cryptocurrency project called Worldcoin. The aim of the project is to confirm whether a user is human or a robot, and potentially pave the way for an “AI-funded” universal basic income. Despite concerns about privacy and data security, Worldcoin has attracted over two million sign-ups across 33 countries. The company aims to get billions of people to scan their irises to prove their humanity. While the crypto coins are not available to US citizens due to regulatory concerns, Worldcoin has seen significant interest from Europe, India, and southern Africa.
Worldcoin’s process involves scanning a person’s face and iris, with 25 free Worldcoin tokens awarded as a reward. The project has faced criticism for its tactics in poorer nations and concerns over data security. Privacy experts worry about the potential misuse of sensitive data collected from iris scans, although Worldcoin claims no data is stored. Vitalik Buterin, co-founder of Ethereum, expressed excitement about the project but also cautioned about its potential pitfalls and the concentration of power in Worldcoin’s hands.
Twitter founder Jack Dorsey tweeted apparent criticism of the project, describing its mission as “cute” with a dystopian warning. Sam Altman welcomed criticism, acknowledging the ambitious nature of the project and the need for experimentation to drive progress.
Despite the mixed reception, Worldcoin’s full launch has seen thousands of people queue at scanning sites worldwide. The project plans to deploy 1,500 orbs in locations globally to continue gathering scans. While some participants see the potential for the tokens to increase in value, others are skeptical about their long-term prospects.
Worldcoin’s dystopian approach to identity verification and the potential implications of large-scale iris scanning have sparked both interest and concern within the cryptocurrency and privacy communities.
Source: BBC
LongNet: To 1 Billion Tokens and Beyond
LongNet is a new version of the Transformer model that enables the processing of extremely long sequences of text, with a scalable window size of more than 1 billion tokens. The researchers at Microsoft achieved this feat of engineering through sparse attention, dilated attention, and a linearly increasing number of attention heads. They also managed to maintain performance on shorter sequences, whilst having a linear computation complexity and a logarithmic dependency between any two tokens in a sequence.
LongNet, developed by Microsoft Research, is an important development in Natural Language Processing (NLP) and Machine Learning (ML) because it addresses the need for models that can handle longer sequences (e.g., thousands of documents) at once without sacrificing performance or requiring ridiculous amounts of memory.
In the upcoming section, you can get a better understanding of the keywords mentioned above. For those interested in a more technical review of LongNet, I (Kourosh) have done deep dive into its technical details in this article.
Transformer Model, Attention Mechanism & Tokens
A Transformer model is a type of artificial neural network (ANN) architecture that is used for NLP tasks. It is based on the attention mechanism, which allows the model to learn long-range dependencies between words in a sequence. For more information, you can read the famous Attention is All You Need paper by Vaswani et al. (2017).
Attention is a mechanism that allows a neural network to focus on specific parts of an input sequence. This is important for language-related tasks, where the meaning of a sentence can depend on the relationships between different words.
A token is a unit of text that is used to represent a word, phrase, or any other meaningful element. Tokens are used to break down text into smaller, more manageable units that can be analyzed by such language models.
Why is LongNet Impressive?
The LongNet model is impressive compared to other LLMs because it can scale sequence length to more than 1 billion tokens with a low computation and memory complexity, which is significantly larger in other models. Its main distinguishable factors are:
- Dilated Attention
- Linear Computation Complexity
- Distributed Training
- Drop-in Replacement
- Applicability for Many Tasks
In his video, David Shapiro explains why LongNet has the potential to be a stepping stone for creating Artificial General Intelligence (AGI) in the upcoming years, as it has a promising future. Feel free to check out his video for some interesting takes on the future of LLMs.
What do you think? Will we be able to input the content of the whole internet into an LLM as a single prompt one day? And if so, how? All this and more in the upcoming post about LongNet, where a more thorough analysis will be conducted on this model. So stay tuned!
Source: LongNet: Scaling Transformers to 1,000,000,000 Tokens — arxiv
End of another kAi Sabancı Newsletter. We hope you enjoyed it. To keep up with the progress in the AI field, please wait for the next issue. See you!
