code like im 5: memory memory memory! (article 2)

ai like im 5

11 min readMar 18, 2024

before i start, a couple things:

this is a machine learning, deep learning, and ai focused coding series.
machine learning coding can be a lot different than software development and engineering, cybersecurity, and the other fields of programming (will take more about this later)
i will be covering some foundational stuff, python and it’s many packages and frameworks!
and in the future c++ and some common ml stuff like docker, sql, and potentially rust and mojo!
a link to my machine learning + deep learning articles which the code will be based on: https://medium.com/@ailikeim5/list/ai-like-im-5-in-order-87ef4064afe8

note: this only scratches the surfaces of things.

want to know the full magic of computers in a fun and awesome way: https://wizardzines.com/zines/integers-floats/

anyways,

last article, i introduced programming languages:

the vessel for translating human instruction into confusing machine code for the machine

and that these were akin to harry potter

the wond channels the magic (communication within the computer)
but without the proper techniques and instructions (programming code), we can’t get far and it can be very destructive!

introducing two main types of modern programming languages:

and that most of the machine learning and ai creation takes place in interpreted languages (python)

because if we had a dilemma like this

there is a clear best choice: spend more time solving the problem and less time implementing the solution.

and although in machine learning, this is highly driven by community support, easy of use, and many things other things

python kinda blows lol

today we will take a deep dive into these two languages, but before anyone should ever code, they should understand a few key programming concepts

memory

you may have heard before that a computer has memory, and although i don’t have time to dive deep into this and computer hardware in general.

memory is a key reason why people say that programming is difficult (me)

when i was learning to code, there was a key block in my learning process.

i didn’t get the how things were actually stored and skipped over it.

and i wish someone gave me a really good explanation on memory first and told me a very important thing before i dove into actual code.

remember two important things

under the hood, computers are just 0’s and 1s
if we are going to store billions of data values, we are going to need a magic translator or magical look up system.

meaning this lookup system is going to be optimized up the wazoo and not going to be invented by an average joe. it’s going to be invented by gigachad programmers who probably spoke better code than english.

memory addresses

i want you to think as a memory address like a present to our computer
it starts out completely empty with nothing inside of it.
there are tons of present, like millions of them
and they are super tiny, meaning we can only store a very small amount of information inside each.

each present is going to have an address, think of like a house address or email address
it is going to be in that fancy machine code that computers love to speak
and so when we want to store data, we essentially look up that address, finding that place in memory, “open” that present up, and store our value as fancy machine code inside it.

we can then

close the present
tag and record the address
and then whenever our computer needs to access the value inside, it will look for the address of the present and then open it up to get the value.

more about addresses

each address is in order… a very simple order (incrementing by 1)

let’s look at a very very very small chunk

note: the 2 and 3 are not actually possible, but important for visualization of the incrementation!!!
don’t get intimated by all the zeros in ones too, you don’t have to know what they mean!

or in poor cs visualization terms:

in my original pictures, the 2 and 3 weren’t possible, but our addresses in programming languages are going to be in another system of conversion called hexadecimal.

where 0x7FFE4A71 is actually 01111111111111100100101001110001

but as you can see, 0x7FFE4A71 is a lot prettier than that weally big number

wow this makes a lot more sense why is it important to store memory in order like this?

well, those gigachad programmers i was talking about came up with a wicked system of storing numbers, letters, and more.
combining the powers of machine code encoding and the look up system
remember… they optimized it up the wazoo, don’t overthink it, your brain might not be wired like that and that’s okay!

if we take these really small presents, we can create some crazy data types.

think of data types like bundling presents together
each present/memory address is going to a 1 byte of memory
a byte is nerd language for group of binary digits
i.e -> usually 8 zeros and ones! (again these is generally the case and it can vary)
keep in mind 1 gigabyte, you’ve most likely heard of, is 1,073,741,824 bytes. yes billion!

data types

here is the general way things work

once again, it gets complicated depending on machine, programming language, and more but we are going to start at the simplest level.

this is a amazing foundation that most software engineers in a bootcamp don’t even get lol.

with 1 present, address or byte we can represent a range of 256 whole numbers or values

although that’s awesome, that’s not a lot of numbers though

why would this be so important?

let’s say “hypothetically”, if we had a system of representing all letters, some punctation, and more with 0 — 128, we can represent them with a single byte of memory.

shocker we do thanks to people like this.

the system is called ascii for you nerds out there!
but we are going to call these characters!

and so if we want to store words, we can bundle together individual presents.

each present is meaningless without the others!
and hypothetically with 1 gb of memory could store a collection of 1,073,741,824 presents, bytes, or in other words a weally weally big essay.

once again, data is just numbers… everything is numbers!

what about bigger numbers though?

so with 1 byte we could encode our machine code into -127 to 128, however that’s not a lot of numbers!

but we can combine two together bytes together, and encode them… we can expand our range!

and what about 4?

with just 4 super tiny presents, we can represent almost all possible numbers we could possibly use
if you ever code, you will mainly work with integers even if you don’t need all the extra space!
these presents/bytes are right next to each other in memory!

if you continued on with 8 bytes, 16, 32, you could expand your range exponentially (by such a unnecessary amount)!!!

if we want to represent digits like this

each present is a part of the bundle
it is meaningless without the others
they are still 0s and 1s to a computer, but our computer can’t translate without the others.

i think it’s really important to go and visualize what this means.

what about decimal numbers?

this system is a bit more complicated…
meaning it is going to have a fundamentally different way of converting from fancy machine code than normal integers.
it is going to be computed a different way.
and because we have to cut off our decimals at a certain point

but again, we can use 1 byte, 2, but the most common is 4!

there is a lot of dweeb magic behind this one!
i.e why we can use decimals and why it can expand to such a larger range than an int!
here’s a dope source for this https://floating-point-gui.de/formats/fp/

and like int, we could continue on expanding our range for 8, 16, 32 bytes expanding precision!

in the future, we will dive into why this is so cool and so so so important in the ai world.

the next simple data type

bool

there are two possible values in a bool.
technically you can use any type, int, float, or any datatype as a bool
so some languages don’t even have a bool.

but there is beauty in simplicity.

combining presents

so hopefully you are understanding a clear pattern

we can take an individual byte that store really simple 0s and 1s

(our computer loves to speak this simple language)

combine them together
and make some really cool things

there’s a lot of power in combining these presents but is there a limit to what we can combine?

the point where most people get lost:

there is no theoretical limit to what these presents can be and how much of them we can combine.
giving us so much flexibility and power.

however, there is a practical one

your computer has a limited amount of memory
you are limited by that particular programming language (and boy let me tell you about programmers)
you are not that guy pal! (you are not smart enough to make a good programming language)

and don’t reinvent the wheel, make your life easier, the concepts in programming are really not that hard!

data structures

each programming language is going to have some core simple data types exactly like or similar to the ones explained above.
these things are called data structures, they are a bit dynamic, one could say…

but data structures are more like this picture

and this sum its up too

yeah we could go on for hours

every programming language has their own way of doing it.
there are a lot of similarities, a a lot of standards, yet there’s a lot of confusion.
because programmers can’t agree on anything

so you might feel like brian at first, trying to keep up with everything in the programming world.

and you might still feel like brian if you coded for 30 years.

and now we can go back to this original image

asking a quick common sense question:

if we have millions or billions of presents in a computer, would it be better to store closely related things near each other or far away?
like the word bozo, should the memory addresses of b, o, z, and o be right next to each other, somewhat close or really far away.

not rocket science!

it is incredibly stupid to store things that are closely related far away from each other!

and now we can introduce our first dynamic data structure

arrays

not only, it is incredibly stupid to store things that are closely related far away from each other!
but thanks to the nature of how things are stored in our lookup system, we can create a data type specifically for storing multiple things together
there is just one limit for now… they must be the same data type, meaning that they must be the same memory size.

note: there are exceptions and work arounds we will highlight in the future, don’t overthink it!

arrays are contiguous or in plain english

each element is stored right after the next
there are no gaps in the memory storage of them.

characters array

if you look at the picture above, we have created a character array
when we combine characters into an ordered sequence like this, we call them strings.

and what’s really cool about these character arrays or strings, because they are a single byte and they are stored contiguous blocks of memory (really close together with no gaps):

if we have the address of the first and we want to access the second third, or fourth element in an array
all we would have to do is add 1, 2, or 3 to that address.
remember, they are incrementing by 1 each time.

this is such a simple calculation and makes accessing elements inside the largest arrays ever amazingly simple

each programming language is going to have their own way of handling strings, how they handle the memory, and more so it can get a bit more complicated. i will be iterating later!

int arrays

let’s say we wanted to store a list of numbers in the same way we did characters.

we run into a slight problem when accessing elements because integers already are a combination of 4 presents/bytes
we can’t just add 1 to access the next element, and so on
and each present is essentially useless without the rest.

but wait, this isn’t rocket science:

there is a clear pattern:

why don’t we just add 4 bytes each time to access the element
i.e → add by the size of the data type each time

so you can see why using the same datatype is so important.

we can do the same thing for float numbers, bools, storing and combining them together.

a great question:

Josh, you said that arrays should be the same datatype, because of the size. floats and integers are often the same number of bytes!

there is one more important thing i have to mention: safety

although the float and integer often will have the same number of bytes, they are fundamentally different things and have processes that are completely different.
it is dangerous, unmaintainable, and stupid to store them together even you could make it technically possible
i don’t think there is a programming language that will let you do it!

in the next article, we will dive deeper into this and some actual code!

so anyways,

this is a really good foundation into memory. there’s probably some stuff i’ve missed and more i’ll get into the next article.

been loving the ufc lately and man…

have a amazing day and