AI basics for programmers 1.

What is AI? How does it even work? Whenever you ask such questions, you’re doused with endless walls of text aimed at technically illiterate people, delving on irrelevant and trivial matters instead of getting on the point. Nobody ever seems to bother to explain it from a traditional software engineer’s viewpoint. Well, here you go.

If you are a programmer, want to learn AI, build AIs, or use them in your applications, here are the main highlights.

To run an AI model on your computer, you need an AI framework. For oldschoolers: an engine. The most popular ones are TensorFlow and PyTorch. TensorFlow also has a Lite version for small devices, such as the Raspberry Pi or the Jetson Nano. These are Python frameworks, and you can extend them using traditional programming methods to leverage an AI’s capabilities. Frameworks also exist in C++ and other languages.

An AI model is basically a data filter. The AI framework acts as an interface to it. The framework accepts your data (text, image, whatever), runs a gazillion of highly complicated calculations based on the model, and returns the result. In fact, this is an oversimplification, as models can do more than just filtering data, but we’re discussing the very basics here.

A framework can work with any kind of model. This means you can use models for very different purposes (LLM, image detection, speech synthesis, etc.) with the same framework. If we will ever have a GPI (General Purpose Intelligence), which can perform any task, these frameworks would likely be able to use it too.

An AI model itself is pretty much a mystery box. Unless you’re a mathematician, and intend to build a career on it, you probably won’t need to mind what’s inside. This stays largely true even when you start training your own AI, but later on that.

Perhaps you already heard that AIs need GPU to run smoothly. This isn’t always true. Often you can achieve pretty good results on a good CPU. A GPU is very useful though, because it can execute multiple instructions parallelly, while a CPU can only execute them sequentially. (It may have multiple cores, but it will never even come close to a GPU’s capabilities.) So a GPU will almost always boost the AI’s speed.

Why almost? Apart from edge cases, it depends on the GPU. To achieve meaningful acceleration, you will need a CUDA-enabled GPU, which means an nVidia product. CUDA is a programming layer which allows direct access to GPU resources. You probably won’t need to know more about it, as it’s the AI framework’s job to talk to CUDA. More accurately, to CUDAs, or CUDA cores, which are individual units on the GPU chip. When you look at the specifications of a GPU card, you may find “1024 CUDAs” or a similar figure on the list. This indicates the number of CUDA cores. The higher the number, the more capable the GPU is.

It is to be noted that while CUDA is a dominant technology in the AI domain right now, it’s not the only one. There are other, similar ones, such as OpenCL, which is an open source equivalent for any platform.

However, CUDA is not the single most important metric you must look for. The following parameters are equally important:

• Clock speed: indicates the overall execution speed of all cores.
• Memory size: it’s a good idea to have more GPU RAM than the size of your AI model.
• Memory bandwidth: the speed of data exchange between the GPU and the RAM is obviously important.

When purchasing a GPU card, just make sure all these parameters are through the roof. I mean, if your family name is Musk, Rotschild, Bezos or something like that. Actually, you don’t need to aim for the moon. For most purposes, a mid-range gaming GPU will do just fine. Selecting the optimal GPU for your use case is an art in itself.

Alas, you can’t really connect a GPU to a Raspberry Pi, but there are some solutions in the single board computer domain too, from GPU add-ons to dedicated single board computers such as the NVidia Jetson Nano. They aren’t very well suited for large models, such as LLMs, due to their memory limitations.

I think we can end this article here. Let me know if you liked it, and whether I should continue in a similar fashion. My intent is not to teach AI technologies in intricate detail, but to provide a general explanation to programmers like me, who may be confused about all the buzz around AI technology.

In the next post, I will show you how to download an existing model, how to run it with ollama, then Tensorflow, how can you find anything on Hugging Face, and what is Hugging Face to begin with. Then, in some later posts, we’ll delve into the basics of training, fine tuning, quantizing and other black magic.