Code is a Network
The structural implications of AI
Before networks, computers were standalone machines and before the internet, computer networks were limited to individual organizations.
The internet connected all computers, smartphones put them in the hands of everyone. This rewired global communication & interaction and as a result, restructured society itself to become one where anyone could talk to anyone, anywhere. For the first time, there were, in effect, infinite communication channels.
The previous structural change to the flow of information, triggered by the printing press, disrupted the Western world. It reduced the control of the Catholic church over the dissemination of religion as it allowed the bible to be mass produced and read by anyone in any language rather than via a priest, in Latin, from a hugely expensive hand written volume. This structural change in information control triggered the Reformation and more than a hundred years of war till society settled down into a new stable equilibrium. When it later extended to other mediums such as music and theater, via records, movies and TV, it created a culture dominated by popular music and entertainment rather than elite art forms that predominated till the late 19th Century. This happened, since, with near unlimited records or movie reels vs individual opera or theater capacity, the purchasing power of the masses was greater than the elite. The structural change from one-to-a-few to one-to-many was now aligned with economics and consumption in free-market economies. While the Bolshoi, became the primary theatre of the new Soviet capital, dishing out opera and ballet to the elite, ironically it was capitalism that delivered rock n roll and Hollywood films to the masses.
If printed text was a shift from one (the author) to a few (a congregation) to one-to-many, the development of the internet represented a shift in communication from one-to-many to many-to-many. This was therefore a bigger structural change than the one that had triggered the Reformation. The rise and falls of individual platforms such as Twitter or Threads or whatever comes next is not interesting in itself and specific platforms or their owners aren’t a cause of culture wars. The whole notion of social media itself is the cause as social media is the model of communication in a many-to-many world.
The internet has thrown us all together as one and while that may be a noble end goal for humanity, diversity and fragmentation can be a feature in whatever Goldilocks balance exists between decentralization and unification. Societal unrest catalyzed by the internet will only stabilize when social groups re-Balkanize so that nuanced debate happens in siloes and tribal affiliation (we are a tribal species) sits in largely isolated pockets based on identity rather than geography. Overlaps between multiple identities will be the glue that holds society together and any online public square for everyone to interact which survives, will by necessity become a bland, lowest common denominator space of harmless, dopamine driven entertainment. In this respect, it’s no surprise that TikTok is emerging as the digital opiate of the masses. Given the West’s history of involvement in actual opium addiction in China, it doesn’t need a conspiracy to see why China would be disinterested in helping the West with controls on TikTok that it has rolled out for domestic consumption.
The Internet has shown the structural importance of networks when it comes to communication, but this is now extending to information processing itself, via AI.
Our brains are networks that take inputs from the senses and pass them through a series of connected neurons to deliver an output action, thought or memory. The neurons are effectively analog-to-digital gates that either (1/yes) pass signals on or not (0/no) based on thresholds of signals from the sum of the input signals for the connections.
Even small brains are capable of amazing things. A spider can create a web of intricate mathematical geometry, but it takes a larger brain to explain the mathematics of a spider web. Similarly, our brain intuitively understands the mathematics of how to hit a tennis ball and our primate brain would have been capable of it, but it took millions of years for a communicable understanding of the language of mathematics to emerge. The codified language of words, logic and mathematics were a self emergent phenomenon of extremely large networks.
In a computer, brain-like networks can be represented as input weights (a number that represents a connection or initial variable) added and processed by functions (neurons) to produce output weights that are then end results or inputs into new functions. That’s all a modern AI system is, it is a network, just a big one with lots of weights.
The connections and neurons are in multiple layers (rows of neurons) rather than complex webs like in the brain (largely for ease of retracing steps and knowing what’s going on) and the functions are non linear or the layers are effectively equivalent to a single one and you haven’t really got a network, just a linear regression (best fit number for the multiple when outputs are a simple multiple of inputs).
In a non linear relationship, the output is the square of the sum or something more complex, not just a number plus a direct multiple of the sum e.g. five plus three times the sum of the weights. The output can still be binary, 1 or 0, fire or not fire, like in brain neurons. If you were to plot the output of a biological neuron as a function of its input, it wouldn’t be a straight line. Instead, it would look something like a step function: below a certain input level (the firing threshold), the output is zero (the neuron doesn’t fire), and above that level, the output suddenly jumps to one (the neuron fires). This type of function is nonlinear because it doesn’t consistently increase or decrease at a constant rate.
Weights are setup with initial values and later adjusted by training.
The individual neurons can all have the same function but have individual biases (numbers that reduce the output — effectively creating a ‘firing’ threshold) which are automatically adjusted during training.
Training is the automatic adjustment of connection weights and neuron biases by continuously saying whether an output is better or worse. e.g. input of a picture of a cat outputs the word cat. It is typically made possible using two algorithms: one which finds which weights need to be tweaked most (backpropagation — called that because adjustments between layers have knock on effects so you work backward to fix) and one which knows whether they should be increased or decreased (gradient descent).
The inputs and outputs of the model are numbers. But in the real world, inputs are pictures and text. Embedding or feature extraction is the process of mapping inputs to numbers and output numbers to images or text etc.
The astounding thing about these networks of numbers and functions is that if they are big enough, they perform a bit like the human brain, with all its foibles, and that things like the rules of mathematics or language are an emergent phenomenon. Not only are they an emergent phenomenon but the emergence seems to spontaneously happen at a critical threshold of complexity much like a phase change between water and ice. Such a phase change was achieved in November 2022 with the release of ChatGPT, creating a quantum leap in technology innovation. The results of this and potential implications were immediately visible and comprehensible to a mainstream, non technical audience of millions, precisely because the innovation was to create something very human like.
But if the results were human like, AI software is not computer like. From the 1940s computers have been based on logic, specifically boolean logic represented by a series of valves (gates) in electrical circuits that mimic logic: ‘If this then that’. You actually only need one type of gate a Not And (NAND) which basically says: if ‘it’s raining’ and ‘I have an umbrella’, then ‘not wet’ (false) otherwise ‘wet’ (true). Amazingly you can build a universal machine (a computer) with wiring that consists just of wires and NANDs.
Also, amazing is the fact that in a parallel universe where von Neumann hadn’t stepped into the fray, computer might not have been made of logic gates but neural nets from the word go.
But till now, we were in a world where software was encoded in logic and therefore the things that computers could do had to be codified and logically understood. There were many vulnerabilities that this produced from people exploiting further logic to undermine these logical processes with logic gaps and bugs, but at least the computers were built on top of the higher level phenomena of consciously expressed logic that had emerged in complex human brains. The new vulnerabilities in systems will be for the opposite reason — that there is no code. AI might turn out to be a weapon of mass disruption as we have no understanding of its logic.
Now, absolutely everything is different. We talk of AI algorithms, but apart from training algorithms, AI systems don’t have any algorithms in the meaningful sense, they have settings and they are networks.
The world of AI is completely and utterly different, we have come full circle and reverse engineered humanity by taking the language of logic that was a self emergent phenomenon of brains as complex networks and built systems using this logic that were able to create brain like networks as a new self emergent capability. We now don’t know whether the universe operates under the universal language of logic and mathematics or whether these themselves are a self emergent phenomenon of the universe as a giant network of complex interaction.
As a network at the level of an application — an infranet, modern AI systems’ capabilities represent a structural change to civilization more profound than the internet which was a structural change which in turn was more profound that the Reformation.
There is no code and there are no applications, there are just networks (AI models) connected to networks which are in turn connected to networks (the internet) processed by networks (our brains).
