Emergent Conformity
A discussion about simulation and an introduction to the end
Transitioning
We started working with various methods and models for visualizing data to generating fake propaganda images. We even spent some time generating lyrics for an R&B pop song. All of our pivots and “mini”-projects have made us look at the piece from different technological lenses. This included thinking about how we could start synthesizing these technologies and techniques to better express aspects of the Chinese Social Credit system. What technologies would provide the best canvas to illustrate ideas like ideological vagueness and the credit system’s call for behavioral conformity?
A new direction materialized when our artist Ying brought up the idea of simulation as a possible technology. She had been recently looking into various simulation based works and became intrigued with the idea of a constantly evolving nature and its potential for resulting emergent behavior.
Thematic Development
During our research we sought to explore thematic relationships surrounding the social credit system and various aspects of simulation. This naturally prompts several questions: What are we simulating? What are we trying to accomplish with this technique, and what behaviors are we trying to observe? For our piece, we addressed these questions by exploring these concepts with respect to computational modeling of psychological phenomena, machine learning, and algorithmic simulation.
- Stockholm Syndrome through Computation Modeling
Stockholm syndrome is a psychological disorder coined after a six day hostage situation in Sweden. It is a phenomena where hostages begin identifying with their captor, and eventually become amicable towards their aggressors. This idea intrigued us as a potential metaphor for how individuals can grow to identify and become grateful for pervasive technologies — like surveillance and censorship — even though they infringe upon the rights of the individual. This metaphor maps directly to how citizens of China recognize the level of censorship and monitoring that the government imposes, but view it favorably as a means of preserving their well-being and prosperity.
In order to create a simulation we need to have an underlying understanding of the processes related to Stockholm syndrome. While Stockholm syndrome has been observed in numerous instances, there is no absolute consensus on what mechanisms cause the transition of a victim to ultimately side and collaborate with their captor. Researchers have sought to break it down into physiological, psychological, and context dependent factors (see relevant papers 1, 2, 3, 4). With this in mind, we aggregated and examined the features that psychologists have proposed can cause such a transformation. With this in mind we discussed how we could meaningfully represent these factors into a virtual simulation.
2. Learning Conformity through Machine Learning
In the Social Credit System, you are given a score that represents how “good” of a citizen you are in society. As you can imagine, depending how you define good can radically influence the resulting behaviors. Likewise, depending on how you frame the rewards and punishments you create a means to control the behavior of a society. Given this framing, we saw a very clear parallel to the idea of Reinforcement Learning.
At a high level, machine learning approaches such as Reinforcement Learning use a defined reward to guide the behavior and actions of agents (agents typically being virtual simulacrums of people or machines) with respect to some goal or task. The power of Reinforcement learning is that rather than explicitly instructing the agent what behavior is desirable at each decision step — like supervised learning — you specify a set of high level goals, associated rewards, and punishments to guide the agents behavior towards a desired effect. For example, say we made the overall objective of our agent winning a game of Starcraft. To help guide it to that outcome you might specify rewards for ‘trying to stay closer to allies’ (1,2,3) as a means to help it learn how to win. Eventually, over millions of simulated Starcraft matches the agent begins to shape their actions to conform to these goals. The exciting part about this process is that the agent’s actions towards achieving this goal are emergent and can produce unexpected results. This idea of guidance under loosely or vaguely defined goals embodies how we could re-imagine the idea of simulation to generate interesting behaviors.
3. Algorithmic Conformity from Emergent Behavior
One distinguishing characteristic that we wanted to address was emergent behavior in the context of herding. Controlling and influencing how crowds move and cluster can create interesting effects. We viewed this aspect as having potential to thematically express the use of the social credit system.
Abstractly, the idea of emergent behavior is that the outcome of a system cannot be understood by its individual parts, but, rather through their interaction. One can imagine this effect in flocking, or swarming behavior.
This effect can be created algorithmically through algorithms such as the ones used in the Game Of Life and Flocking with Boids. By defining a set of rules for each individual, you can get group behaviors that are unexpected or unintended. For example, flocking behavior results from three simple rules:
- separation: steer to avoid crowding local flock-mates
- alignment: steer towards the average heading of local flock-mates
- cohesion: steer to move towards the average position (center of mass) of local flock-mates
None of these rules enforce the individual to flock, but because of these rules a form of unintended herding occurs. Relating back to the social credit system, we thought about how the rules enforced could cause unintentional changes to the social dynamic of communities within China.
Conclusion and Going Forward:
By examining and synthesizing different themes and technologies, we have realized that the concept of people’s behaviors evolving over time is incredibly compelling. Throughout our experience so far, we have tried to abstract aspects of the social credit system by using different machine learning and visualization techniques. Each approach helped us think more critically about what our narrative is and how technology can enhance how it is expressed. Next time we will begin to illustrate how we applied simulation to these themes and the technologies involved.
