Published in

Geek Culture

# Key Highlights

1. How data is stored- To understand how these things work (and why they won’t replace people), let’s first understand how models process inputs. In the simplest case (and is most often), input is stored as a vector (which is a fancy math term for a list). Each element of the vector represents a particular feature. Think of as a feature as a characteristic of your sample. If we were creating a data vector of you, things like height, weight, numberOfGrandmasPunched, likesChocolateMilk, etc would all be features.
2. Comparing Similarity between 2 data points- Remember, each data point is stored as a vector. To compare similarities, all we have to do is compare the distance between the vectors. That is where the formula above becomes helpful. It is called Cosine Similarity. There are other distance metrics you might want to use, based on your needs. As with every other decision, there is a tradeoff. I’ve covered Cosine Similarity in more detail here, for those interested. For our purposes, the important thing to note is that Cosine Similarity can give us the similarity between 2 vectors.
3. What does this have to do with Big Models- Reasonable question. To oversimplify a behemoth, the models take your user input and try to generate an answer they think would match it well. The match is determined by the similarity (or the complement, which is called disagreement). They do this very well because of the enormous resources sunk into training them. They’re able to build very fine search spaces based on this, which is not the case for standard models.
4. Why these models will not take over your jobs (utility)- Now for the bit that these influencers conveniently leave out (or haven’t considered). These models’ pattern match existing data. They can’t work with new inputs. For example, if I created a new library, the vaunted GT-CoPilot would struggle with it (it even struggles with existing libraries). If all you did was create art/scripts from older data, you would never create new characters that reflect the tastes of your contemporary audience.
5. Why these models will not take over your jobs (engineering)- These models have also gotten some engineering problems that make them unusable at scale. For one these are extremely expensive to run. This makes them impractical at scale. Secondly, they are very weak to perturbations and can be broken very easily. Not good for anything that has to be deployed into the real world. I’ll be doing a more detailed into Github CoPilot soon, so make sure you stay tuned for that.

# Reach out to me

--

--

## More from Geek Culture

A new tech publication by Start it up (https://medium.com/swlh).

## Get the Medium app

Deep Insights about Artificial Intelligence (AI), Machine Learning, Software Engineering, and the Tech Industry. Follow me to come out on top