AI Top-of-Mind for 7.10.24- MLSec
Today: MLSec, Small Language Models, AI in education, how to find GPUs, and a novel approach for content creator compensation
Top-of-mind is AI security. With all the investment and interest in AI, what are we doing to secure everything from the network to the training databases. ‘CB Insights’ details the ‘MLSec’ space, with the following definition:
The machine learning security (MLSec) market provides solutions designed to protect machine learning models and algorithms from adversarial attacks, data poisoning, model evasion, backdoor injections, and other cyber attacks. Vendors offer a range of products, including intrusion detection systems, adversarial defense systems, secure machine learning frameworks, and anomaly detection tools.
Startups covered include HiddenLayer, Patronus AI, Arthur, and others. The research then goes on to what trends are driving the MLSec market:
- Cyber threats becoming more sophisticated and frequent as a result of AI.
- Enterprises’ widespread adoption of generative AI tools, introducing new vulnerabilities like employees using third-party AI models.
- More stringent security regulation, like the SEC’s recent rules governing cyber risk management and incident disclosure.
With all the focus on the largest of LLMs, is there a simpler path? I’ve written a bit about Small Language Models (SLMs) and their size vs performance, as well as their suitability for running on edge devices like smartphones. Here, the ‘Wall Street Journal’ offers greater detail. One advantage pointed out is the lower cost of responding to queries, related to coverage of AI energy consumption. And this reminds me of the recent Apple Intelligence announcement where queries are directed to the proper SLM or LLM. A good quote from the article:
· For many tasks, like summarizing documents or generating images, large models can be overkill — the equivalent of driving a tank to pick up groceries.
· “It shouldn’t take quadrillions of operations to compute 2 + 2,” said Illia Polosukhin, who currently works on blockchain technology and was one of the authors of a seminal 2017 Google paper that laid the foundation for the current generative AI boom.
· The credit-rating company Experian EXPN 0.06%increase; green up pointing triangle shifted from large models to small for the AI chatbots it uses for financial advice and customer service.
· Once trained on the company’s internal data, the smaller models performed as well as large ones at a fraction of the cost, said Ali Khan, Experian’s chief data officer.
We’ve all read about the use, and misuse, by students of AI in education, but here is another take. As reported by the ‘NY Times,’ Los Angeles has scrapped a program due to financial and delivery problems by the startup it engaged. Good lessons for the future. From the article:
· An A.I. platform named Ed was supposed to be an “educational friend” to half a million students in Los Angeles public schools. In typed chats, Ed would direct students toward academic and mental health resources, or tell parents whether their children had attended class that day, and provide their latest test scores. Ed would even be able to detect and respond to emotions such as hostility, happiness and sadness.
· Los Angeles agreed to pay a start-up company, AllHere, up to $6 million to develop Ed, a small part of the district’s $18 billion annual budget. But just two months after Mr. Carvalho’s April presentation at a glittery tech conference, AllHere’s founder and chief executive left her role, and the company furloughed most of its staff. AllHere posted on its website that the furloughs were because of “our current financial position.”
Continuing on the education front, a report by the National Education Association (NEA) on the use of AI in the classroom. The report does a good job of splitting the problem into different domains such as reactive, predictive, and generative AI, and looks at looks at the technologies impact on students, teachers, and even institutions. Some recommendations from the report:
And how do hot AI startups access the GPUs they need for LLM development? They get a little help from their friends, in this case the VC firm Andressen Horowitz. ‘The Information’ reports on their stash of over 20,000 GPUs for use by their portfolio companies, GPUs that could be hard to obtain. The approach is also helping the firm engage potential startups. From the article:
· Andreessen Horowitz started providing access to its GPUs earlier this year to a small number of startups, in exchange for equity. The startups also pay to rent the chips at a discount to market price, the person said.
· It’s not clear whether Andreessen Horowitz has purchased the chips or is renting them, which is typical for AI developers. Forbes earlier reported the VC firm was negotiating with chip providers to set up a compute program.
Lastly, on creative, thoughts on compensation by David Gilbertson as he proposes a path forward in the midst of bots that don’t click on ads. His novel idea is to bring the chatbot vendors (i.e., OpenAI) into the loop to provide payments to website content developers. There are a bunch of questions he addresses and offers a technical solution that could be workable.
