Five Acts of Dreams between Hong Kong and New Haven

Mechanical Eye Final Reflection

Prelude

Life is like a dream. My move from Hong Kong to New Haven to attend architecture school at Yale has been a hazy dream. This journey inspired by my residual muscle memory of my childhood bedroom in Hong Kong while walking half asleep to the bathroom here in New Haven takes on the challenge to identify the ambiguous duality of our memories of space, especially those that we inhabit.

The use of numerous digital tools and methods in this class, as the “mechanical eye” looking through the lens of the aforementioned narrative attempt to recreate this experience to the visitors. The repeated digital manipulation of the site in the past week introduce us a different way to conceptualize our relationship with space, especially ones we envision in dreams.

The five acts following this prelude becomes a wild dream of mine that I wish to walk you through the dreamscape of my two bedrooms.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Act I — Sounds of Sleep

Sound is an inalienable in our perception of space. The semi-lucid state of mind in dreams composes ambience sounds into strange music of the dreamscape.

As one of the most influential figures in contemporary music, John Cage has pioneered a myriad of non-canonical ways in making music. “Everything in the world has its own spirit which can be released by setting it into vibration.” A statement that was proclaimed by Oskar Fischinger, which was later recalled by John Cage on his interaction with the abstract animator as his assistant (1). It is perhaps then the experimental composer inside Cage was born, everything became music to the ears. Indeed, his work seeks outside of the boundaries of the staves of Western sheet music. His dynamic music scores (fig. 1), often consisting of powerful curves and geometries, forces us to rethink music beyond that of the creation of a conscious composer. Music of Changes (1952) and 4’33” (1952) manifest the aforementioned idea by taking in the mundane data of probability and sounds of our body into the realm of music. What we sense and conceive as the world is transformed into a collection of rhythmic sounds we call “music”.

fig. 1 — Music score of John Cage’s “Fontana Mix” (2)

Following the line of thought of John Cage, this part of the project seeks to mimic the synthesis of music in dreams. Sensor Play, a motion sensing recording app on iPhone was used to track my walk from half-lucid walk from the bed to the bathroom in New Haven. Strapping the phone on my wrist, the phone became an extension of me and recorded all my swaying movements. The unstable gait translates to the zig-zag lines plotted on the gyroscope graph (fig. 2), which are then exported as a spreadsheet of data (fig. 3).

fig. 2 & 3— Graphs and spreadsheet of motion capture data of my sleepy walk from bed to bathroom

This data set of nearly 10000 data points was then imported to TwoTone, an online app that translate data into sound and music. Similar to Changes, which the numbers from dice-throwing are matched to musical notes, the x, y, z positions as recorded by the gyroscope are also matched to create a three soundtracks. Each of the soundtracks corresponds to one instrument (marimba, glockenspiel, and oscillator) in the C melodic key (fig. 4) to produce an eerie sounding piece of music that becomes the background of the dream.

fig. 4 — Screenshot of TwoTone showing the process of tuning in the generation of the background music from the motion capture data

fig. 4–30 mins of resulting ambient music without breathing sound

4’33” tunes our ears to the often overshadowed sounds of our body and environment. In dreams, the rhythmic sounds of our breathing adds to the overall acoustic experience of the dreamscape. The soundscape, as a record of the midnight bathroom walk, is now complete.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Act II — Blankets of Sleep

The second part of the project deals with the visual aspect of the dreamscape. Continuing on the idea of ambiguous duality of the two rooms in Hong Kong and New Haven, fragments of each room begins to infect their counterpart in the creation of a dreamscape that blurs the initial recognition of rooms.

The work of Christo and Jeanne Claude, as well as that of Haruhiko Kawaguchi, explores the pliability of fabrics and plastics in the molding of people and objects. Wrapped Reichstag (1995)(fig. 5), arguably the most significant piece of work of the duo, reveals and highlights the basic form and proportion of the architecture through its accentuation created by the folds of the fabric (3). Kawaguchi echos with Cristo and Jeanne Claude in that he vacuum seal couples and families with their house in Flesh Love All (2019)(fig. 6). The loving image of a family with their domestic dwelling is wrapped and condensed under the single entity of the plastic vacuum sealed bag (4). Both works speak to the creation of spatial ambiguity and duality that my project seeks to create through draping.

fig.5 — Wrapped Reichstag, Berlin, 1995 by Cristo and Jeanne Claude, photo taken by Wolfgang Volz (5)

fig. 6— Flesh love all by Haruhiko Kawaguchi (Photographer Hal) (6, 7)

The draping of the blankets of meshes, created from the previous photogrammetry scans of the two rooms, starts with the building of a blank framework of the rooms. A largely faithful recreation of the two rooms with all its furnitures dematerialized were built in Rhinoceros 3D (fig. 7). They were then repositioned and connected by the closet, which acts as a space-time portal between the two rooms which are physically thousands of miles apart. As part of preparation for the draping, the photogrammetry mesh that were repeated manipulated in the previous weeks was cut into pieces using Zbrush (fig. 8). Pieces of furnitures like the bed, bedside table, cabinet, and wall fixtures like windows were cut out as individual blankets to be draped onto the blank model.

fig. 7— White framework model built to act as the foundation and base for the draping. New Haven room on the left and Hong Kong room on the right.

fig.8 — Reconstituted mesh from previous exploration that was being cut up in Zbrush

fig. 8— Process of cutting up the photogrammetry scanned mesh into individual meshes of the furnitures.

After all the preparation work, the blank 3D model was then imported into Maya to be draped by the meshes of furniture blankets using nCloth. As the project tries to blur the perception between the two rooms, decision was made to swap the meshes of the two rooms to be draped. In other words, the furniture meshes of the Hong Kong room would drape onto the dematerialized model of the New Haven room, and vice versa. As the meshes of furniture began to fall onto the rigid body of the framework, they began to take on the shapes of the framework (fig. 9), making them bearing the quality of the two rooms at once. As this process of draping went on for the two rooms, a dreamscape of ambiguous duality begins to form. The strangely familiar form of each room takes on the texture of the other. The recognition of the rooms by the mind is now delayed due to the confusion of the misaligned imposition of the textures and the forms. These blankets mimic as fragments of memories of the other room as one walk through rooms in their dreams.

fig. 9 — Snapshots of the zCloth draping process in Maya

fig. 9 — Video showing the draping process looking from Hong Kong bedroom into its counterpart in New Haven

fig. 9 — Video showing the draping process looking from New Haven bedroom into its counterpart in Hong Kong

fig. 9 — Final static site looking from New Haven after draping rendered in Maya

fig. 9 — Final static site looking from Hong Kong after draping rendered in Maya

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Act III — Ghost of Blankets

Even in dreams, the space around us is never static. This rings even truer for fragments of memories that exist in our unstable subconscious mind that are constantly distorted by time. The ghost of these fragments, as blankets of furniture mesh, were animated in Unity to mimic this distortion.

Dreams are nonsensical yet a reflection to our subconscious mind (8). As a reference to the New Yorker article read weeks ago, the meshes were made floating up and down alongside the rhythmic breathing of the ambience sound. The movement of the mesh also conceptually echos with the slow rise and fall of the chest during sleep, further enhancing the embodiment of the dream within the virtual space.

The ghosts of the blankets is also interactive. They slowly get out of reach and become intangible as one try to catch it. As metal and muscle memories slowly fades away with time, they become farther and farther away from the dreamscape. In order to recreate this experience, a unity script was written to move the meshes up every time one tries to touch it (fig. 10) The blankets of me worry fragments slowly rise to the heaven, becoming intangible to touch yet still barely visible from afar.

fig. 10–3 minutes exploration of the dreamscape, showing the floating meshes and the interaction with them

fig. 10–Mesh flying up into the heaven after repeated attempts to touch it

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Act IV — Forest of Dream

Previous reflections on Rosalind Krauss has proposed the placement of the dreamscape beyond existing structuralist field of sculpture. The rooms of the dream do not float in the universe of nothingness, they instead are embedded within a landscape of dream that resembles that of a foggy evening inside the deep forest of both Connecticut and Hong Kong.

The rooms of the dream were superimposed onto the landscape made during the past exploration of nCloth draping as a technique of finding the boundary of the site (fig. 12). In Unity, the two virtual artifacts of the dreamscape intersect one another. Parts of the landscape were allowed to invade into the original boundary of the rooms (fig. 13). The erosion of the man made by the nature reinforces the constant distortion of space in dreams. The artifacts intertwine with each other to form a singular whole of the dreamscape which is to be experienced as the dual embodiment of both the tales of domesticity in New Haven and Hong Kong.

fig. 12 — Previous exploration on using draped nCloth as landscape

fig. 13 — A view showing the invasion of landscape into the room

Adding to the mystic and blurry atmosphere of the landscape, environmental fog was added to the site in Unity. As one walks through the dreamscape, the fog hides the totally of the space, only to be revealed once become close. Spatial intimacy in dreams is represented by closeness in the virtual dreamscape.

Like everything else, the fog in dreams is not static, but rather fluctuates when one falls in and out of consciousness. Using the particle system in unity, generators of fog were placed strategically to allow fog to both hide and deep into the bedrooms. The constant blurring of the rooms, landscape and the forest concludes the making of the virtual dreamscape which now can be experienced and embodied through screens and spectacles (fig. 14).

fig. 14 — Final images of dreamscape from inside to outside of the rooms. Rooms, landscape and the forest are constantly being blurred by the fog

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Act V — Dream of Artifact

Coming to a close of the tale of dreams of the two bedrooms, like waking up from a dream, some kind of physical trace is left to be desired. Intrigued by the conversation on materiality of a physical artifact of the dream with the guest critics during the final presentation, I have decided to 3D print a model of the rooms in white PLA plastic.

Even with the great might of 3D printing and the amazing plasticity of the material, it does not render the full resolution of the digital site. In moments where blankets of meshes were draped onto the framework model, the blankets become rather chaotic strands of plastic adhered to the more regulated framework of the rooms (fig. 15). Nonetheless, the loss of fidelity during this translation from the digital to the physical in certain sense represents the loss of memory of a dream when one wakes up. The materialized version of the dreamscape is never the same as the version in our heads.

fig. 15 — Top view of the 3D printed artifact of the dreamscape

fig. 15 — Angled view of the 3D printed artifact of the dreamscape (top), Close-up view of New Haven room (bottom-left) and Hong Kong room (bottom-right)

“What a dream.”

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

References and Further Readings:

1. John Cage, “An Autobiographical Statement,” (speech, Dallas, TX, April 17, 1990). John Cage Trust. Link: https://johncage.org/autobiographical_statement.html
2. Jimmy Stamp, “5 1/2 Examples of Experimental Music Notation,” Smithsonian Magazine (June 5, 2013). Link: https://www.smithsonianmag.com/arts-culture/5-12-examples-of-experimental-music-notation-92223646/
3. “Projects: Wrapped Reichstag,” Christo and Jeanne-Cluade, accessed December 23, 2020, https://christojeanneclaude.net/projects/wrapped-reichstag?view=info.
4. Kala Baraba-Court, “Photographer Hal Vacuum-packs Entire Families… and Their Homes,” Plain Magazine (September 20, 2019). Link: https://plainmagazine.com/photographer-hal-flesh-love-all/
5. “Projects: Wrapped Reichstag”
6. “Flesh Love All,” Flesh Love All — Photographer Hal, accessed December 23, 2020, http://photographerhal.com/flesh-love-all/.
7. Baraba-Court, “Photographer Hal”
8. Stephen Marche, “It’s Dreams that American are Making Sense of Trump,” The New Yorker (March 29, 2020). Link: https://www.newyorker.com/culture/cultural-comment/its-in-dreams-that-americans-are-making-sense-of-trump