The algorithmic turn: what do crowd-sourcing, photogrammetry, geo-tagging, and mixed reality mean for the visual?

Point cloud generated as part of photogrammetry process
Photosynth aggregation of 2D images into 3D space
Keiichi Matsuda’s augmented reality visualization, Hyperreality

In The algorithmic turn: photosynth, augmented reality and the changing implications of the image, published by William Uricchio in 2011 in the Journal of Visual Studies, Uricchio explores the algorithmic incursion into representation using two case studies: Photosynth¹, the online crowd-sourced photogrammetry service; and Augmented Reality technologies. At the heart of both systems are algorithms that are location-aware and determine representation and perspective to suggest an emerging visual regime.

With Photosynth, location-tagged photographs are aggregated from the images that anonymous users have posted online, to algorithmically generate a 3D geometry of the a real scene that users can virtually navigate through. This near-seamless space is not a representation of a single perspective, but an aggregation of multiple perspectives (the system allows up to 600 different images to generate a single scene) through a crowd-sourced authorship. With augmented reality applications, the algorithm can recognize and map physical space, and superimpose virtual graphics. In this case geo-location is tied to the user’s perspective accurately (or the angle of the display device).

The nature of the algorithm in creating such a system challenges western norms of representation that have dominated the modern era. The unified subject-object position is subverted, the validity of traditional 3-point perspective is displaced to representation through many perspectives, representation is not tied to the synchronization of a single position, perspective, and time, there is an expanded model of authorship, the concept of the determining frame or screen is rendered obsolete, and, traditional 2D images are morphing into 3D interactive, navigable, geometry, dramatically changing the experience of representation for an audience, or user.

The nature of algorithmic intermediation in these technologies points to having social meaning and value in the same way that the creation of perspective did during the Renaissance. The social consequences of the invention of the Cartesian coordination system were significant, and Uricchio’s paper proposes that we should likewise consider the social consequences of new algorithmic technologies today, in the creation of meaning and value.

Uricchio points out that the rise of the algorithm only really took place in the latter half of the C20th, due to the advent of computers and the need to process large amounts of simple data. Prior, mathematics’ influence was through the arithmetic percision: Euclidean geometry, the Cartesian coordinate system, the Mercator map projection of the world, which all directly influenced perspective, spatial understanding, representation, and how we understand our place in the world. Whereas today, mathematics is primarily driven by algorithms, and Uricchio asks what social implications can we detect in this early period of “the algorithmic construction of the image.” (p.25)

Uricchio refers to Canaletto’s painting Piazza San Marco with the Basilica, from 1730, as as example of the regime of representation of the modern era with its stable, precise, fixed, and singular point of view. In contrast, he describes how Wikipedia has emerged as a crowd-sourced collective point of view, by algorithmically aggregating many individual authors, bits of information, and opinions. “Rather than relying on the expertise and reputation of one known individual, the reader must take a more active role in making sense of the ensuing composite of anonymous voices, in assessing it, in moving across its links to pursue additional information.” (p.30) Wikipedia, as an established and widely known means of the algorithmic construction of meaning, is aligned with Photosynth as a similar example of the transition of meaning-making, and is in sharp contrast to Canaletto’s painting.

Whilst in Photosynth, the user is physically static, their experience is composed of an algorithmic assemblage of plurality (points of view, authorship, time), one moves within the image, and in AR, the user experience is embodied and constant, and their experience as a viewing subject is algorithmically modulated through a bridge between the virtual and physical, where the image is geo-located to the user and in motion.

In the example of Photosynth and augmented reality, the construction of meaning takes place via descriptions in the form of data tags that are either assigned to images of the real world, or assigned to the real world through AR recognition systems that also can be linked to “deeper data repositories (websites), and even other links (telephone connections), reaffirming the assignment of meaning to place. AR systems effectively overlay the viewer’s access to the physical world with specific (and selectable) grids of signification.” (p.33) Uricchio concludes that there is “a fundamental reworking of our position as subjects vis-a-vis the world in this new algorithmically enabled era, (…) [and] may well in itself suggest aspects of an emerging new order, one in which procedural intermediation repositions the old certainties.” (p.33) There are challenges to the very subject positions and logics that so much of our current social order is based on, and the valuation of site alongside the representation of information is markedly changing.

Whilst in the modern era, mathematics was driven by precision arithmetic that led to a subject-object binary and is still the basis of our current social, economic, and cultural order, “the algorithmic turn remains rooted in human experiential and semiotic practices. (…) But as we explore the new affordances of the algorithmic, and as our capacities to deploy them grow in tandem with the progression of Moore’s Law, we might also begin to reflect more critically about the differences in emerging modes of representation.” (p.34)

¹ Photosynth has since been discontinued as an online service, likely due to the prevalence of new photogrammetry and scanning systems.

Works cited:
Uricchio, William. The algorithmic turn: photosynth, augmented reality and the changing implications of the image, 2011, Journal of Visual Studies, 26:1, 25–35