Together with my WOWlab team I am working on an art project Bilibin XR.
Bilibin XR is an interactive folk tale for AR glasses.
In this publication, I will talk about creating an image of a Window into a Fairy Tale.
About the children’s imagination
In the child’s imagination the world of a fairy tale is inseparable from the child’s real world.
What happens in a fairy tale is amazing, but real. It is a part of the surrounding world. It seems that if you just open a window, an amazing story full of fairy magic will unfold.
About the ways Bilibin transferred a Fairy Tale in his works
The great artist Ivan Yakovlevich Bilibin formed the image of a fairy tale for millions of people through his illustrations.
Bilibin inscribed illustrations of the fairy-tale world in the painted frames.
The frames became magic windows that invite us into the wonderful world.
Like the frames of a comic book, Bilibin used frames to give additional context and emotion to what is happening in the illustration in their margins.
Bilibin’s works continue his deep research into ancient culture and symbols.
Bilibin’s works inspired a whole line of illustrators and film directors. Dozens of films of fairy tales, which are watched by millions of people, were shot based on Bilibin’s style.
I still remember the anticipation of a fairy tale that I had in my childhood when I saw an image of the opening of the ancient wooden window. This is how a series of fairy tales, designed into films, used to began. The window was opened and the storyteller told the tale.
About Fairy tales among the people.
The structure of a fairy tale is such that we participate in a fairy tale action just by listening to it. We go down the path with a hero, thinking about how we would act in each fairy tale situation.
Here the hero stands in front of the stone at the intersection of the two roads and the listener of the tale, just like the hero, chooses which path he would take.
Notice how the frame around the illustration is illustrated. Through symbols and small details, we get a new emotional experience from Bilibin’s illustrations.
About a fairy tale and a Myth
Actions associated with a myth contain a ritual.
The ritual was perceived by the people as indistinguishable from science. Just as science describes the physical properties of the world and the impact on these physical properties, so the ritual directly affects the fabric of being.
James Fraser, in his study of cultures in The Golden Bough, describes how the culture of mythical consciousness perceives the actions of rituals and their influence on the laws of the universe.
James Fraser is a British religious scholar, anthropologist, ethnologist, culturologist, folklorist and historian of religion, a representative of classical English social anthropology, who made an enormous contribution to the study of totemism, magic and the transformation of religious beliefs throughout the history of mankind.
Fraser analyzed ceremonies in many cultures in order to categorize them and explore their meaning and where the ideas in the ceremonies originate from.
One of the striking examples is the group of rituals for invoking rain.
Excerpt from the book “The Golden Bough. Study of Magic and Religion “ by James George Fraser.
“In a village near Dorpat (present-day Tartu) there was a need for rain, usually three men climbed tall fir trees in a sacred grove. One of them, in imitation of thunder, beat with a hammer on a pot or a small keg; the second, in imitation of lightning, struck sparks from the burning smut, and the third — he was called the “rainmaker” — sprayed water from a vessel in all directions with the help of a bunch of branches. “
Earthly actions affect heaven.
Fraser’s idea of part and similarity describes how people with mystical thinking saw the influence of their actions on the heavenly laws.
If something is similar to one another, then their influence is interconnected. So the image of water pouring through a sieve is like rain, which causes rain.
Heaven in the form of a man. A man in the likeness to heavenly.
Excerpt from the book “The Golden Bough. Study of Magic and Religion “ by James George Fraser
If the deity finds pleasure in mercy, forgiveness and purity, and not in bloody sacrifices, singing hymns and smoking incense, then it is best to please the deity without prostrating before him, not singing praise and not filling temples with expensive offerings, but being filled with purity, mercy and compassion for people. Indeed, in doing so, people imitate, as far as their human weakness allows them, the perfection of the divine nature. This is the ethical side that the Jewish prophets tirelessly inculcated, inspired by the noble ideals of divine holiness and kindness.
Actions of Heaven affect the Earth.
Rite as a Game Mechanics
In our story with augmented reality glasses, the rite acts as a mechanic of interaction.
What happens in the fairy tale world depends on the actions of the player. The actions of the player are formed into a ceremony — the purpose of which is to influence the currents of the world.
Excerpt from the book “The Golden Bough. Study of Magic and Religion “ by James George Fraser.
“Prophet Micah exclaims: “O man! you are told what is good and what the Lord requires of you: to act justly, to love works of mercy and to walk humbly before your God”(Micah 6:8). And in later times, Christianity drew the strength with which it conquered the world, from the same high idea of the moral nature of God and the obligation imposed on people to conform to it.”
In the fairytale world of interactive storytelling for AR glasses, the player’s actions influence the course of the story. A wonderful experience that augmented reality glasses technology provides is the experience of the power to change reality.
When the world of ideas appears through the augmented reality glasses the experience of influencing the world of ideas is manifested.
Excerpt from the book “The Golden Bough. Study of Magic and Religion “ by James George Fraser
“In India, up to the present time, the great trinity of Hinduism — Brahma, Vishnu and Shiva “obeys” the brahmans, who, with the help of their charms, exert such an effect on the most powerful deities that they are forced in heaven and on earth to humbly carry out the orders that their master-sorcerers will be pleased to give. There is a saying in India : “The whole world is subordinated to the gods; gods are subject to charms (mantras); and charms to the brahmans; therefore the brahmans are our gods.”
The greatest power is to control your attention. Contemplating the heavenly radiance in every moment of clarity.
Creating the reality of the Good.
About Breathwork Practices
A player wearing augmented reality glasses approaches goodness and moves history through breathing practices.
In the first chapter, in which the Player meets the Firebird, the sun moves in the sky through calm, measured breathing. Augmented reality visualization is changing. Animation occurs as you go deeper in meditation.
For seven years I have been practicing meditation for up to 5 hours a day. Part of the meditation experience was with a neural interface connected to my head.
The device reads brain waves and depending on the phase of concentration, changes the sound. If thoughts are restless, it rains. The quieter the mind the clearer the weather and the birds sing.
In the story for AR glasses, the deepening of meditation affects not only the sounds, but also in the visual environment.
In a fairy tale world, events move when the sun goes down. The player has to guard the magic apples when interacting with the story. According to the story the apples in the garden of the Earthly King are eaten at night by some uninvited guests.
Breathe calmly and day will turn to night. Inhale = exhale.
Meditation practice governing the world of fairy tales.
Within the framework of the tale of “Ivan Tsarevich, the Firebird and the Gray Wolf” two prince brothers fell asleep without waiting.
The idea of sleep and wakefulness is embedded in the history of the Good. One must arrive in clarity for the Heavenly good to descend on earth.
Fraser’s idea of a part of the Whole.
Fraser’s idea of the piece explains the reasons for the influence of earthly actions on heavenly laws. The influence of the parts is attributed to their nature as part of the One. Just like the influence on the hand has an effect on the body. So the action of the hero affects the heavens.
As an example of how a part affects the whole, Fraser describes voodoo magicians who, getting some part of a person, for example, his hair, got the opportunity to influence a person.
The actions of the hero have an impact on the world — because the Hero is part of Heaven.
The hero is the repository of the heavenly spirit.
In Greek mythology, a hero capable of passing the path is a demigod. He is related to divine power. Like Hercules, the son of Zeus. Kinship is a metaphor for having a part of the heavenly spirit within. When the heavenly soul is manifested, then it is power that is consistent with the power of heaven.The power to influence the world.
The rite within the framework of the myth serves as an action renewing the course of history. A driving force across the boundaries of different worlds. For example, the rite of passage that separates childhood and adulthood.
Realization in your own way.
The hero’s journey in The Power of Myth by Campbell is the journey of renewing the energy of the world.
The world is built not only for personal benefit, but also for the insight of the common good. The common good unites the world and is above the world, outside it. The hero on his journey sees the good and brings it back to the tribe.
Meeting the Firebird on the way is a metaphor for the insight into the radiance of the common good. The hero of the fairy tale has not yet established himself in the radiance, but was able to see the sun reflected in the Firebird as in an earthly incarnation. Having received part of the good in the form of the Firebird’s feather, the hero goes to the Shining.
The path of the Hero is the path to the Shining of Good.
Rite and Fairy Tale
The difference between the Rite that exists within the framework of the myth and the fairytale action is that the fairytale Hero goes against the established rituals.
For example, Ivan Tsarevich, Saves the Tsarevna, who is being prepared as a sacrifice to the lake.
For the villagers, in their world order, Ivan Tsarevich is a character who threatens the very order of the world. The saving the Princess can disrupt currents of the world.
The prince, in the framework of the fairy tale, is a hero who rescues the princess. Ivan Tsarevich acts contrary to the rite.
The intersection of ritual and fairy tale is one of the sides that we want to reveal in our interactive story for AR glasses.
The actions of the rite are part of the script. The mechanics that influence the narrative are the actions of the rites.
Fairy tale interactivity for AR glasses.
Bilibin XR contains 7 chapters of an interactive fairy tale.
Each chapter is a complete independent part of the story with its own augmented reality interaction mechanics.
Each chapter can be traversed in different ways. The story changes based on the actions of the Player.
In the first chapter of the Bilibin XR project, the player meets the Firebird. The player receives a task from the Father, the King of the earth to protect the magic apples.
In the tale of “Ivan Tsarevich, the Firebird and the Gray Wolf,” the Tsar has three sons. Two sons who received the order to guard magic apples at night fell asleep. They couldn’t keep their attention .
One of the experiences in the first chapter is the experience of Expectation. The Player has enough peace of mind to breathe and watch the sun go down. Behold the beauty. A reverse impulse to a culture of consumption in which there is a dynamic change of picture. The first chapter deals with the very existence of the moment in which attention occurs.
In an interactive fairy tale for AR glasses, the Metaphor of expectation is conveyed through the practice of breathing.
Take nine calm breaths and you will calm down. Calm the waves and see what is happening in its full depth and beauty. Like the sea containing the sky.
WOWlab team has already created XR projects for meditation practices. In the Flow project, Stone Balance, VR Shishkin, Apple XR meditation, we have carried out in-depth research on creating a meditation experience using XR technology and neural interfaces.
In the Bilibin XR project, the Player is free to make his own decisions within an interactive fairy tale. The player can show calmness and follow the actions of the rite, or he can eat apples and then the story will take a different path.
All our stories and rituals are based on deep ontological research by authors such as Fraser, Propp and Campbell who deeply explored the culture of mankind.
About portals in augmented reality
Through the interactive mechanics of augmented reality glasses, we will unfold a fairy-tale world around the player.
The player will take his or her own heroic path through magic adventures.
The window is an important symbol in a fairy tale adventure. Augmented reality portals are one of the most spectacular augmented reality mechanics.
When you look at another space unfolding outside your window, it is amazing. The mechanics give a surprising effect. This is how our perception works. If we find out that there is a large space in something small we do not expect this and therefore we are surprised. As if we were opening a box, and there was a window to a new world.
Windows in augmented reality are broadcast through glasses to real walls around the player.
The portal is an elegant solution to the technical constraints of augmented reality glasses. All glasses on the market have a FOV which lets you see augmented reality.
By composing the narrative through the portals, the Player always sees the full picture of what is happening. Technical restrictions do not interfere with the immersion process.
About the concept of the Magic Window. Concept via Magic Leap.
In order to know the required scales of 3D objects in augmented reality, I made a sketch of the window dimensions using the AR glasses. Putting on Magic Leap glasses, I marked on the walls of my apartment what size a magic window in augmented reality would be.
I used the drawing function and outlined what size the window should be in order to get into the FOV.
I outlined blue spots, thinking that by opening the window, the Player will see the space of the Magic sea.
In a distant kingdom in a distant state, beyond the sea there is a magic city.
For 5 years WOW lab team and I have been doing art XR projects for world museums.
The best experience is created by designing projects directly in XR. More iterations not on paper but directly in the environment where the script will take place allow for more opportunities to make additional settings in the project so that the player’s feelings are triggered more clearly.
Think about it. In classic cinema the director makes us cry when the Titanic sinks by manipulating our attention and arranging the narrative. This magic of cinema comes from understanding and carefully planning artistic methods in the film.
Augmented reality glasses are a new medium for storytelling. A reality in which the Player’s attention is free to be anywhere.
My background is making AAA games. In a new reality (VR / XR / AR), many new ways of storytelling are added. The way the story takes place, when it is in your physical world, but you are able to build additional images and meanings over the physical reality.
We have devoted a lot of time and effort to in-depth research in the creation of experiences in a new reality (VR / XR / AR) during our five years of work in WOWlab.
We teach the XR experience design methodology at the British School of Art and Design.
As part of the program, we have already trained over 200 designers in three years.
Based on the Bilibin XR project, we are creating a new course on creating artistic experiences for AR glasses.
Our art project acts as a platform where we implement all the accumulated knowledge of design experience.
About drawing the Magic Window concepts with a street artist.
In describing the work on the Magic Window, I want to mention a street artist from Georgia. Georgia is a wonderful country. Tbilisi, its capital, looks great with its distinctive architecture.
I met an artist who grew up around these houses. I told him about the fairy tale for AR glasses and he drew the first pencil sketch.
Stages of work on the concept of the Window.
First, we prepare a mood board. We build a style, indicating which images we refer to when working with a window image.
The main ideological references are the works of Bilibin.
Bilibin deeply studied wooden architecture. Cultures of working with wood.
We note the use of graphic shapes and patterns. Patterns existing around the windows of platbands intersect with the style of patterns around Bilibin’s illustrations.
It is important not to go over to stylization in the Khokhloma and Zhostovo styles, more modern adaptations of the folk style.
In the Billibin XR project, we focus on ancient ornaments in wooden architecture dating back to the Bronze Age when people began to create houses and decorate their lives.
It is important that the tradition passes through the forms as a continuous motive. In culture the window was perceived as a portal between the worlds and in ancient times the application of symbols on the window carried a protective force transmitting the connection of the worlds.
The window is a sacred image. A way for the spiritual to enter the house.
We mark the general points for working out the image of the window.
When creating a window, it is important to consider how it participates in the story. What are the options for the Player to interact with the magic window that appeared on the real walls? It is important to adapt the graphic elements so that they can complement each other’s narrative and fit into different physical environments in which the project will be launched on AR glasses.
We consider how all the elements of augmented reality will look together. When preparing 2D concepts, you need to understand how it will look in 3D.
For the convenience of distinguishing details at the concept stage, we define soft fill light for the graphics and assume several angles from which the window will be seen.
We draw the concepts on a light neutral background, without using dramatic harsh lighting and deep falling shadows that could prevent us from seeing the shape.
By opening a window that appears on the real walls, the player will open a portal to a fairy-tale world.
We take into account that we need to understand how the window will look in closed and open form. We immediately think about the design of a handle that a player will grab using his real hand to open the window.
We think over the presence of a drawn symbol on the window and assign solar symbolism to it.
We fix a few thoughts on the concept of the window
- The platband can be patterned, but not thick. Not overloaded with details and not to push, not to distract a lot of attention. Framing the portal we look through the window into a magic world. The window is an intermediary between our world and the amazing world of fairy tales.
- The upper and lower part of the window should not be made straight, it is better to add a pronounced geometry there. That way the window becomes a balanced independent object. Having implemented it in 3D — it will be possible to place it in augmented reality.
It is important to see the difference between styles in order to move towards the ancient images of architecture, and not, for example, wooden classicism and wooden Art Nouveau.
Working with Window silhouette
Which window option do you like more?
Let’s make a combination of several ideas. We liked the birds. The window is used for the chapter of the interactive story in where the Player encounters the Firebird.
The symbol of the Firebird is the heart. The heart pattern is expressed in the fiery plumage of the Firebird. It is also a symbol of how the radiance of the common good penetrates into the incarnation of the spirit. Through the Heart.
We use the shape of nemgo birds with a shift from the center (as in option C) to make the overall silhouette more expressive. The silhouette of the birds sitting on the window began to be seen more clearly.
Let’s turn the birds to each other. For the image of birds, let’s take the symbol of birds found in a pattern in Bilibin’s illustration.
Let’s complete the image of birds on the window. We will add the images based on the seal of Bilibin. Bilibin noted a series of printed fairy tales with his illustrations using his special seal. Each book begins and ends with such a seal with birds eating a single fruit.
Our composition becomes more dynamic after flipping the birds. It gets more activated. Let’s add an apple as an image of a fruit to the center of the composition.
Inside the fairy tale about “Ivan Tsarevich, the Firebird and the Gray Wolf”, the Firebird flies into the garden at night to eat magic apples.
For the shape of the window we used the element of repeating squares also used in Bilibin’s illustration along the edge. It is the closing element between the vertical and horizontal in the illustration. Also in the window concept, we note this pattern for the upper and lower parts.
We determine the options that can be used in both central and lower parts of the illustration. For the lower edge we considered adding an image of the fruit from Bilibin’s seal. As the second option we thought that the lower part can be used as an illustration of where the action takes place. By placing an illustration of the landscape to the lower part we give to the Player additional context on where the magic window is located and the action takes place. Where the Player opens a magic portal.
We determine which parts will be in 3D — a wooden shape, and what pattern will be drawn on the wooden surface of the window.
So the central shutters and the lower part of the windoware used as a place for drawing. The combination of 3D forms and illustrated elements additionally connects the image of the Window with Bilibin’s works.
We complement the outline of the illustration that will be on the central part with the window shutters.
Working with color
When starting to work with colors we use large spots in order to look at how color can become part of the narrative. Let’s see how the color can unite the composition.
By choosing green as the base color we emphasize that the action outside the window takes place in the garden. We emphasize the vegetative essence.
We emphasize the shapes by choosing red. See how different color options flow around the shape. Notice how the overall perception changes in the red version.
By choosing a palette with a Blue filling we emphasize that the action takes place at night.
When changing to a palette with a darker blue and caramel pink shade our perception of the image also changes. See how twisted columns go up on the sides of the window. They lead our gaze to the symbol of the heart on the top.
It is also interesting to work with plain filling options. This painting option refers to the images of a wooden ancient jewelry box.
We look at the stars in the image of a window working more with the accents. The same star patterns are on the frame of Bililbin's illustration where Ivan Tsarevich Meets the Firebird.
In the image of the window, we transmit the radiance of the star through the two lower columns, then we reflect in the lower part of the illustration where the patterns of the star are reflected in the starry sky.
The stars are moved to the central part, where the patterns of stars in white are visible in the sky above the firebird.
There is a mystery in the ongoing episode of an interactive fairy tale. The hero goes to protect the magic apples by the decree of the Father of the King of the Earth. The player does not know who is eating apples in the garden at night. The two sons of the king fell asleep before they found out who was eating apples at night.
Using dark blue shades in the window frame, we will emphasize the metaphor of a mysterious night. We will add gold accents in order to express the image of apples.
Which painting option do you like?
We detail the painting.
We are working on the details. We make the columns thicker at the base. We shift the birds’ tails on top towards the window to create a general silhouette.
In the upper part of the window we add a small dome. In culture, the dome is an symbol of the heavenly kingdom. We draw the sky and the golden glow of the stars on the dome.
Gold highlights magic apples. The apple is used as an image of the fruit. Inside the fairy tale story, a single tree unites all the worlds.
The branches spread out in the radiance of the common good. Fruits have appeared on the branches of a tree growing throughout the worlds.
The magic story happens to the Player inside an interactive story in augmented reality glasses.
The world tree also grows through its reality. The branches of the tree from the fairy tale sprout into the Player’s room.
The world tree sprouts branches into the player’s reality. A magic apple appears right in front of player wearing augmented reality glasses.
In the image of the magic apple, we have transferred part of the wooden architecture. How wood carving works.
We are working on the details of the image of a magic window.
At the bottom of the window we have a drawing on a tree that gives the player additional context. Where the events take place. And we see the scarlet sun setting behind the fabulous city.
The night in the fairy tale story comes because the Firebird descends to earth to eat magic apples. The fruit of the world tree gives the Firebird the power for rebirth and a new day of Rising Good.
Through the colors in the image we tell our story. We are showing their unity by reflecting both the sun and the Firebird in scarlet color.
The development of an interactive story depends on the Player’s actions. The world tree has sprouted into his reality and he can come into contact with the fruits filled with the radiance of heaven.
Inside the interactive fairy tale through augmented reality glasses, we will give the player the choice to follow the rite and save the heavenly apples or alternatively eat them. The player can reach out to the shining apple and make his own choice.
At this stage, our main goal is to partner with companies that produce augmented reality glasses.
We will adapt the interactive fairy tale for the glasses of our partners and show the technical and artistic maximum possible on the device.
If you can help us with contacts or you are interested in participating in the development of the project:
Write to me, Pavel Sovushkin via personal mail: Sovushkin@WOWlabXR.com
On Twitter, I show the stages of work on a project:https://twitter.com/PSovushkin
And Discord where we show work on art objects and interactions for AR glasses: https://discord.gg/vJ2XgGvP
Subscribe to follow the development of the project. We will be glad to have any contacts that will help in the development of the project Bilibin XR for AR glasses.
Thank You.
