An algorithm is a function of a set of actions and rules in a particular context with a particular logic and control mechanisms, deployed in designing technical and digital tools from light-switches to smartphone applications to AI. A nuanced and accurate example of algorithms today is the GPS and navigation systems that direct drivers through an optimum path decided by the coded logic of the software. In broader terms, an algorithm is a set of step-by-step procedures required to produce the desired output from a particular input. It represents a system of possible connections between possible sets of intermediary steps to arrive at a designated goal. The passage from a step out of a set of steps to another step out of a set of steps is a connection, and the chance of ‘success’ (a.k.a. ‘1’) or ‘failure’ (a.k.a. ‘0’) of the connection between particular steps from each set of steps are represented by a calculated probability.
Genealogy of the term
The English word algorithm derives from Arabic via Greek and Latin. It contains a genealogical relation to the Greek word arithmós (αριθμός) via the Middle-English word algorism, which, unlike its conventional association with the term “number”, was endowed with a meaning of ability to recognize, conceive, count, and assign a name to an object (Nussbaum). Algorithm’s mainstream meaning, i.e., stepwise rules and processes, emerges from this relation. The word algorithm derives from the long name of the mathematician Abū Jafar Muḥammad ibn Mūsā al-Khwārizmī, whose two principal manuscripts on mathematics introduced many of the key methods of algebra (derived from the Arabic word al-jabr). His first work Al-Kitāb al-Mukhtaṣar fī ḥisāb al-jabr wa-al-Muqābala (The Compendious Book of Calculation by Restoration and Balancing) found its way tho Western Europe through the Moorish Spain (Crossley and Henry, 1990: 106). The Arabic adjective on the title of the book, ḥisāb, is also suggested to be translated as arithmetic. In English, the first use of the term appears on Chaucer’s Canterbury Tales as “augrim”, and the current use as ‘algorithm’ began in the 18th century. ‘Algorithm’ and ‘arithmetics’ have been used side by side without confusion since the Middle Ages (Karpinski, 1914: 708).
The Seductive Business Logic of Algorithms | Data Driven Investor
Certain machine behaviors never cease to amaze me. I'm astounded by their ability to learn from their accomplishments…
Until the mid 20th century, the term ‘algorithm’ was not used in the standard sense, and the arithmetic dimension of ‘algorithm’ was not common. Because al-Khwārizmī’s other (untitled) manuscript on Indo-Arabic numbers provided the Western Europeans with the ‘Arabic’ numerals, arithmetic and Arabic numeration system became synonymous until the second quarter of the 20th century (Crossley and Henry, 1990: 104). Arabic numbers were preferred to be called “the numbers of algorism”.
This association with numbers, arithmetic and algorithm fulfilled its signification with a range of connotations that are fundamental to the understanding of the genealogy of algorithms. First, the word algorism has a close affinity with the word zero (Smith and Karpinski, 1911: 58). The word zero comes to English from the Arabic sifr “cipher”, through the Sanskrit sunya-m, meaning “empty place” (Ibid.). In algorism, the word “cipher” is used alternatively with zero, even with any number in a way as to substitute for algorism (OED Online). Because one of the original meanings of the word sifr is “empty place”, zero also carried a sense of occupation of this redundant place by a “non-existent” person in a derogatory sense (OED). The mainstream definition of the word cipher today, a secret code to be decrypted, emerges along with its historical meaning.
The meaning of algorithm in the 20th century has thus been bifurcated. First, via sifr(cipher)-to-algorism, algorithm meant both revealing or concealing (excluding or not-counting). This incorporates complicated meanings; the contemporary meaning is used in representing the real or promising to represent what is not yet invented but yet to be discovered. Then, second, ‘algorithm’ gradually came to be conceived as a system that refers to a set of step-by-step procedures required to (re)produce and reveal certain and given knowledge about the world; recognizing, counting, categorizing, apprehending and naming objects. The former incorporates complicated meanings. Contemporary meaning is used in representing the real or promising to represent what is not yet invented but yet to be discovered.
Steps toward 21st Century: computer technologies
The standard texts regarding the development of algorithms in the 20th century are three milestone works from information studies, neuroscience, and computer sciences, all of which were gathered under the umbrella domain of cybernetics. Hungarian-American scientist John von Neumann, in his lecture at Cambridge in 1935, initiated the concept of storage in computer science. His model of computer, which was going to be the backbone of any computing tool so far, was composed of a CPU for arithmetical functions and a mass memory storage unit that can be randomly accessed for multiple tasks. It was Alan Turing who embarked on von Neumann’s earlier computer model and ideas to come up with his famous thinking machine, going against the claims that a computer can not operate autonomously. Von Neumann was influenced by Turing’s theoretical contribution to his ideas, and he worked on bridging computer and the human brain in his posthumously published book The Computer and the Brain in 1958, in which he made the case that the algorithmic network of this computing machine is a different model of neural networks of the human brain (he compared the similarities and differences between the human brain and computers). In his model there is a central processing unit (CPU) that conducts the work of arithmetics and logic; the memory part stores programs and data; and there are input and output channels. The modern model of algorithms.
Norbert Wiener’s Cybernetics: Or Control and Communication in the Animal and the Machine, published in 1948, embarks on the computing machine, animal organisms and the basics of computers developed by von Neumann and Turing. Wiener’s book was a landmark for many reasons, mostly practical. His most bold move was to subsume cybernetics under neuroscience of animals. In brief, the contemporary meaning of cybernetics emerging from Wiener’s text is the title of the book: the ways in which animals and machines control and communicate, bearing in mind that humans are an animal species. Wiener refers to algorithms only in one part of the book, and he deploys the word in the sense of logic or reasoning. Combined with the title of the book, control, Wiener’s piece was going to give the simple formula clarified by Robert Kowalski, which is used in almost all introductory texts today: Algorithm = Logic+Control (Kowalski, 1979).
Wiener developed an insight into the logic of the ways how animals get feedback from the outside world, encode it through their organic networks (through corporal senses to neural networks in the brain, etc.), then gain equilibria in their organisms. And he proposed this organic feedback mechanism as a model to the development of computing and self-reproducing machines, soon to be known as computers. Wiener believed he was developing a theory of computing machines as self-learning and thinking apparatuses, but, in fact, he produced among the first algorithmic theories of machines and animals. In his concluding remarks, Wiener compared animal organism with the self-regulating market system, or, economic liberalism. This allegory was clear in his mind: control is achieved through autonomously-generated equilibria. Consequently, Wiener conceived of cybernetics compatible with nature, by applying to the logic of self-regulating systems.
Machine learning is a field that lies at the intersection of all categories and forms of algorithms today. It refers to the process in which algorithms establish models to automatically perform certain tasks. A learning algorithm recognizes, processes, and analyzes data, which is designated to fit in human domains and things. It was Alan Turing who proposed a computer model as a learning machine using an audio dataset recorded to tape. The earlier models of learning machines were taken together with computers and AI and relied on the mediation of data between a machine and a human, or the mediation between “intelligences” on the basis of lingual signs (see Claude E. Shannon and Warren Weaver’s (1949)).
Today, these learning algorithms have interfaces that recognize, predict, and synthesize data, and then generate a response in the best (statistical) way possible toward a certain task or goal. This process of object-recognition is known in the literature as the Hierarchical Hidden Markov Model (HHMM) (Markov 1954, Fine et al. 1998; Weber 2002, Chowdhury 2003, Allen 2006). These recognition models deploy statistical, algebraic functions to recognize and predict things. For instance, unlike humans, digital sensors with these recognition models cannot understand an apple at the first look, it has to compare what it sees with the large models of apples in its dataset and then statistically infer that the object is an apple. Another example of HHMM is the one that is deployed in natural language processing systems that parse milliseconds of voices and break down speeches, detect discourse particles or phonemes to compare this with the ones in its large voice dataset. In essence, HHMM is the common denominator for recognizing a thing. This algorithmic process of analyzing objects take place through sets of rules and actions to predict the context of the thing.
At the heart of processing data is the semantic analysis that is supposed to come up with logical interpretations of things. This happens when the Markov Model parses and divides data into their smallest units in order to help the algorithm incorporate a context and content. For example, if an algorithm analyzes the word “cheap” it requires context to decide if the text means calling something “cheap” financially or if it means something banal or cliché. Semantic analysis, ceteris paribus, helps the algorithmic system make sense of the context.
When an algorithm learns to perform the task, such as speaking or driving a car or turning on lights, the act of learning, imitating and uttering rely heavily on limited and fixed datasets recorded and stacked for the use of processors. The task of sharing information is accomplished by algorithmic connection and correlation, using pattern recognizers trained with repetitive patterns through which the algorithmic system learns to perform the task. Algorithms have thus a learned ability to learn and to recognize structures of things. This ability results from the initial pieces of training, initial supervisions, and initial codings conducted by engineers — a required step for them to perform the task. Repetition of learned patterns, and then putting together small pieces to make a meaningful whole, using this larger thing in another structure, and then reiterating this process help algorithms build exhaustive structures of language, driving, flying, cooking, and many more.
Majority of the research in the field of algorithms to train successful products uses these static datasets. Algorithmic systems make use of these datasets, evolve with differences in input, and generates knowledge through large data. Algorithms that use kinetic data, on the other hand, absorb new data by interacting with the external world (humans and things) in particular contexts.
In common cases, algorithmic systems use the static, textualized data to augment their patterns and to become an enhanced system to transform textual data to master the “deep” order and networks of things. The learning process through kinetic data still needs supervision by engineers, bearing in mind that algorithmic agents are incentivized towards a goal to perform certain tasks. In other words, the system is directed, and breaching points between actions and performance goals are fulfilled manually -a “Bananas for Monkey Reward” (Conneau et al. 2017). In some cases, algorithms are left alone after a process of supervision. Some researches emphasize the fact that humans are interactive species and learn things through their interaction and communication with their social network (Bennett et al. 2003; Lidor and Wang 2016).
Research on learning algorithms revolve around these themes of imitation, supervised learning, contextualization, and incentivization, with different treatments of gathering generating and processing data. There is also an emerging, theoretical approach for the ideal algorithm that would no longer need supervision: “Master Algorithm”. As Pedro Domingos (2015) makes clear, the master algorithm is an all-encompassing algorithm that would recognize things, create things, solve problems about things, and self-grow. Facebook AI Research (FAIR) engineers, for instance, expressed in the International Conference on Machine Learning in 2018 that their algorithm project (Mechanical Turker Descent) would self-generate data through its “master” algorithms that do not necessarily need supervision (Yang et al., 2018). These “master” algorithms can interact with the “outer world” and cross-exchange the data they generate from these interactions (i.e., they can communicate with each other and treat each other as what they are taught to be “humans”).
Forms of algorithms
Today, algorithms are vital to the modes of gathering, registering, processing data in computer technologies. In addition to the technical functions of hardwares, algorithms are used in tablet and smartphone applications, computer operating systems and softwares that provide the steps or actions for any tool to work in a particular order in order to perform the goals or tasks. An algorithm, thus, can also be defined as any sets of performances that can be internalized by a system (Minsky 1967, Savage 1987, Gurevich 2000).
The data that an algorithm processes can come from an input, registered and encoded and stored for further processing. The dataset is an integral part of algorithmic performance. This is a computational and statistical process that describes crucial components of algorithms that apply to various possible scenarios. In any case, contextual actions are systematically taken into account. Since any algorithm is a nuanced and articulated set of steps, the system of the statistical and computational process is integral to the operation of the algorithms. The instructions for possible actions are listed in a standardized way according to the control and order.
This common understanding of the standardization of an algorithm assumes the “essence” of its task in concrete and mechanical terms. It needs an assignment, which decides on the variables of the operation, and so, the value of the goal.
Algorithms are represented by various notations, which include natural language processing, web cookies, programming languages and control panels, codes and many more. Algorithmic representation of language is mostly lingual, uses sophisticated languages, and not employed for algorithms of technical tools such as switches. Many kinds of tables and charts in other forms of algorithms, such as control and code panels in programming, use very clear and unambiguous language that can be executed by computers.
Categorization of algorithms
Keeping in mind the logic and control components, algorithms can be categorized in terms of their design, application, and the fields in which they are implemented. First, means of application vary from iterative algorithms that constantly loop in the same way until a goal is achieved to algorithms that operate serially with a single processor, that is, processing one after another, or in a parallel or synchronized manner with multiple processors, that is, multiple and simultaneous processing through various functions to achieve a certain goal. Another application of algorithms takes into account the degree of determination or non-determination. Determination claims to harbor the answer to a problem with precision, while the application of non-determination performs intuitively thanks to its design, by generating insights or using heuristics. Another important model of algorithmic application is conceptualized as “quantum algorithm” that refers to a limitless sequences of actions compared to the restricted sets of steps of non-quantum algorithms (Nielsen, M.; Chuang, I. (2000). Quantum Computation and Quantum Information.). This algorithm can function on a quantum computer that can simultaneously execute more than serial and parallel algorithms.
The second categorization of algorithms is the methodologies of their designs. There are some methodologies that include different forms of algorithms to achieve a task or goal. A comprehensive approach takes into account each possible solution towards the goal, whereas a disintegrative approach might break the problem into smaller pieces and look at minute details. Graphing is another method that uses the enumeration to map things on a graph. Randomization, on the other hand, is also an effective method to achieve a goal when particular, ready-made algorithms are not capable of generating exact paths to a goal (similar to heuristics in the non-deterministic application of algorithms). Designs of algorithms also include the technique of eliminating a task’s complexities by reducing its difficulties into a known paradigm.
A last, but not the least categorization of algorithms is related to the domains of application. Scientific disciplines have their norms, rules, and logics for particular tasks to solve problems. For instance, the algorithms used in medicine can be different from searching and graphing algorithms. It is important to note that scientific domains intersect with each other, and thus, algorithms used in different fields also tend to be related.
Adam, A. (1998). Artificial knowing: Gender and the thinking machine. Routledge.
Allen, J. F. (2006). Natural language processing. Encyclopedia of Cognitive Science.
Bennett, I. M., Babu, B. R., Morkhandikar, K., & Gururaj, P. (2003). U.S. Patent №6,665,640. Washington, DC: U.S. Patent and Trademark Office.
Cater, J. C. (1983). Electronically speaking: Computer speech generation. Sams.
Chowdhury, G. G. (2003). Natural language processing. Annual review of information science and technology, 37(1), 51–89.
Close, A. J. (1990). Cervantes: Don quixote. Cambridge University Press.
Conneau, A., Schwenk, H., Barrault, L., & Lecun, Y. (2016). Very deep convolutional networks for natural language processing. arXiv preprint.
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017). Supervised learning of universal sentence representations from natural language inference data. arXiv preprint arXiv:1705.02364.
Danaher, J. (2018). Toward an ethics of ai assistants: An initial framework. Philosophy & Technology, 1–25.
Fine, S., Singer, Y., & Tishby, N. (1998). The hierarchical hidden Markov model: Analysis and applications. Machine learning, 32(1), 41–62.
Flanagan, James L. (1965) Speech analysis synthesis and perception. New York City, Springer-Verlag.
Greene, R. (1963). Friar bacon and friar bungay. University of Nebraska Press.
Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002). The faculty of language: What is it, who has it, and how did it evolve?. Science, 298(5598), 1569–1579.
IBM (2018). The new AI innovation equation. IBM Blog — available at https://ibm.com/watson/advantage-reports/future-of-artificial-intelligence/ai-innovation-equation.html
Kelly, Kevin, IBM (2018). What’s next for AI? Q&A with the co-founder of Wired Kevin Kelly. IBM Blog — available at https://ibm.com/watson/advantage-reports/future-of-artificial-intelligence/kevin-kelly.html
Leviathan, Yaniv & Matias, Yossi (2018). Google duplex: An ai system for accomplishing real-world tasks over the phone. Google AI Blog — available at https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.html
Lewis, Mike; Yarats, Denis; Dauphin, Yann N.; Parikh, Devi; Batra, Dhruv (2017). Deal or no deal? Training AI bots to negotiate. Facebook Code — available at https://code.fb.com/ml-applications/deal-or-no-deal-training-ai-bots-to-negotiate/
Lidor, Nadav & Wang, Sida I. (2016).Interactive language learning. The Stanford NLP Group, Research Blog — available at https://nlp.stanford.edu/blog/interactive-language-learning/
Lindsay, David (1997). Talking head. American Heritage of Invention & Technology Summer 1997, 57–63.
Michaely, A. H., Zhang, X., Simko, G., Parada, C., & Aleksic, P. (2017). Keyword spotting for Google assistant using contextual speech recognition. In Automatic Speech Recognition and Understanding Workshop (ASRU), 2017 IEEE (pp. 272–278). IEEE.
Oord, A. V. D., Li, Y., Babuschkin, I., Simonyan, K., Vinyals, O., Kavukcuoglu, K., … & Casagrande, N. (2017). Parallel WaveNet: Fast high-fidelity speech synthesis. arXiv preprint arXiv:1711.10433.
Russell, S. & Norvig, P. (2016) Artificial intelligence: A modern approach (Global 3rd edition). Essex: Pearson.
Shannon, C. E. & Weaver, W. (1949). The Mathematical theory of communication. Urbana.
Sighart, J. (1876). Albert the great, of the order of friar-preachers: His life and scholastic labours. R. Washbourne.
Suchman, L. A. (1987). Plans and situated actions: The problem of human-machine communication. Cambridge university press.
Turing, A. M. (1950). Mind. Mind, 59(236), 433–460.
Weber, D. (2002). U.S. Patent №6,499,013. Washington, DC: U.S. Patent and Trademark Office.
Yang, Z., Zhang, S., Urbanek, J., Feng, W., Miller, A. H., Szlam, A., … & Weston, J. (2017). Mastering the Dungeon: Grounded Language Learning by Mechanical Turker Descent. arXiv preprint arXiv:1711.07950.