Paperol; an open writing tool for time-based VR scenarios
For most writers, writing a story for VR is very different from what they are used to. Actions can happen simultaneously, VR can be gaze- or otherwise trigger-based, thoughts can be made audible and a viewer can even get an active role in what he/she gets to see. What do you do when your normal text processing program does not support your new way of writing? You develop your own software tool that does.
Origin
When PIPS:lab created Anyways scenarist Willem van Weelden started writing the scenario in Word. The story contained six characters in a train compartment, sometimes talking through each other, like people do in real life. The story could be experienced from the perspective of any of these characters, enabling six different participants to experience the same train conversations at the same time, but each experiencing different inner thoughts of their own character to be heard on top of that. Needless to say, Willem had a hard time getting all of these dialogues and inner thoughts structured together in Word. Keez Duyves, conceptual creator of Anyways, suggested to try out Excel instead.
Now Willem could use the grid of Excel to separate time from the different characters, each with their own inner thoughts. Still, Excel is not made to write text. Also, there were a lot of other requirements that were not met by Excel when using it to write for VR. Keez decided he’d have to make a software tool himself to fix the problem: Paperol.
Development
The first version of Paperol was literally made out of paper. In a workshop where this version was used, sentences on snippets of paper were put on a table. Keez used a laser pointer to show the passing of time from left to right, while students read aloud the dialogues. The latest version of Paperol is still based on this principle. Paperol became a digital, very open, time-based writing tool. By pressing the spacebar a cursor will start moving left to right to give a representation of time passing. When writing with this tool, one can use horizontal lines to represent a separation of spaces, of characters or of anything else that makes sense to the writer. Vertical lines represent certain specific moments in time, like the end of a scene or the entrance of a new character to the scene. Arrows can be used to represent movements of characters between spaces, the most prefered gaze path of the viewer, choices a viewer has in experiencing different parts of an interactive story, or anything else the writer might come up with to use it for.
For instance, if you’d use lines to represent space in a 360 video, you could put in three horizontal lines. That way you’d have four parts; representing front, left, right and behind you (or north, east, south and west). Vertical placements of text within those areas could even indicate a more precise representation of the locations of events or characters while they are speaking their lines. Arrows could be used when characters move from one place to another or as options for different paths they can take.
Vaperol
Keez is also working on a version of Paperol that can be viewed directly in VR. This enables the user to import the Paperol file we were just imagining into ‘Vaperol’ (working title). You could make Vaperol understand the representation of space and let it ‘wrap’ your original Paperol file around you in VR. This way all things written happening in front of you, will be written in front and all things written to be happening behind you, will be written behind you. Eventually you should also be able to manipulate your text in Paperol VR, to use the extra dimension of space you have there. This would turn the tool into a way to sketch for VR as well.
The fact that Paperol is an ‘open’ tool means that there are lots of ways a writer could implement the tool when writing for VR. Each VR story is unique and can represent a VR scenarist with a new writing challenge, that might be overwon with a different use of this tool. There is no wrong or right way to use it, as long as the story written is in some way time-based. The Paperol 2D-space and the Paperol VR 3D-space will also be able to relate to each other in different ways. This means that also if you choose to use Paperol very differently than just described in our example, you would be able to make a 3D representation of that in Paperol VR, just the way you’d like to.
Use case: Time Machine
PIPS:lab is planning on making an Anyways 2.0, which means that Pips:lab scanarist Willem van Weelden will get to use and thereby test all the software tools Keez has developed. But until then, it is very valuable to see as much other use cases as possible.
As I (Nikki van Sprundel) was writing this blog, I started to understand Paperol much better. But I still had a hard time imagining how it would work for a 6DOF project. Because of this, I decided it would be a good idea to use my own VR project as a use case (Time Machine). This project was almost done when I did this, which made it easier to sketch it out. It also made it easier to explain in this blog what I did, because I can show you what it actually looks like in VR as comparison. I started out by sketching on paper, just so I was not restricted by what was or was not possible in Paperol yet. This was my first sketch:
Time Machine contains five scenes and takes place in four different spaces. One of the objects seen in those spaces is a time machine. The viewer (in this sketch a circle with a cross inside) starts inside of that time machine, surrounded by a pink empty void. The time machine transports him/her from that empty void into the hallway of Rena’s house.
From that hallway the viewer can walk out of the time machine into Rena’s room. At a certain moment, the viewer will be transported from Rena’s room to Mount Fuji and eventually back again. In the end the time machine will come back to the hallway and the viewer can walk inside again to read the credits. What I found out by making this sketch, was mostly that I missed a distinction between what movement to another space is automatic and what ones are initiated by the viewer. I tried to mark two different possible paths a viewer could take, by drawing in the long pencil line as seen in the picture. This didn’t give me the desired effect, so I made a new sketch:
What I did differently here, is that I crossed out a room when it is not available for the viewer to be in. I’ve also tried to find a way to show how long a character or object is present. I did this by underlining the object and letting that line go on if the object keeps existing. The arrows now mark the path the viewer is taking and I’ve used dotted lines to show the viewer can walk past that line.
When drawing, I used a few things that are not yet possible in Paperol: dotted lines, crosses and a way to show how long an object was present in the space. My aim was to find a way to translate that into a version Paperol could already make. I used arrows instead of lines, because I knew lines were still in development. All things I knew were not (yet) in development, I did not put in. This is what I came up with:
The end result actually looks far more simple and clear. Without the dotted lines it is easier to see that you can walk around between the two spaces. The fact that in one of those spaces there are no characters or object placed, even indicates that it is likely that this is where the viewer would be. Therefore I felt I could also leave out arrows to mark a path. One thing that is still missing would be an indicator for the time that an object or character is present in a space. Although, not having that indication in there makes the total file easier to read. My conclusion is that it might not be necessary to have this option in Paperol. It would be of more value when putting this file into Vaperol, to then have the option in VR to mark these in- and out points in time.
Student use case
One of the filmacademy 4th year scenario students just started writing a scenario for a VR project, with which she and a few other students will graduate at the Filmacademy in 2019. She was struggling with finding a good tool to write it in, so she was very happy when she heard about Paperol. We invited her to the PIPS:lab working space to show her Paperol and Vaperol. By then she had already taken place in the third workshop as well, which was about using Dialogus. When starting to use Paperol, she first struggled with importing text because she had to start by converting her .docx file into a .txt before she could import it into Paperol. After that she struggled with choosing how that text should be placed in Paperol. She felt the need for more tools, like lines and different text colors, so she could make a distinction between sound and visual aspects. She says this prevented her from being able to make a clear structure of her story. I also think it has something to do with the story not being structured enough in her head yet to make the translation of that to Paperol. She did decide she wanted to keep using it, but it should take her some more time before she actually will have a result of her story into Paperol.
Next we put her into Vaperol with a part of her text imported into the empty VR space. The first thing she realised in there, was that she had imported whole sentences and that these were too long and therefore hard to read. Next time she said to only want to use short pieces of text. For now however it was already too late, as she couldn’t edit her texts in Vaperol, just move them around. After trying multiple things, she finally sorted out her text in a kind of ‘tunnel’ structure, instead of all around her. The reason for this was that it was the most logical way to have some sort of timeline represented in VR. By walking ‘through the tunnel’ you could experience the time passing by. At the beginning of that tunnel she placed texts indicating what side of the tunnel stood for what place around the viewer normally in VR. This way she could place all her text inside the tunnel, having the tunnel represent the complete use of space in VR. Also for Vaperol the student expressed her need for more tools. She missed being able to draw arrows and (again) to have her texts colored, so she could make a distinction between audible and visual aspects.
This case study is not done yet, as the development of these students their projects has just begun. They did decide they want to work with Paperol and Vaperol. The idea for this is to already start working with the Paperol-Vaperol workflow, although Vaperol is still in development. The fact that Vaperol does not contain a timeline yet will mean that the IMVFX students who work together with the scenario student will do ‘blocking’ in Unity by themselves instead. This way, the scenario student can sketch in Paperol and this sketch can then be used as a base to start putting placeholders (blocks) into VR on a timeline, so the students can test out the story in VR even in an early writing fase. As the project develops, these blocks will get replaced by real assets.
Future Development
Future development of Paperol will likely hold the possibility of importing pictures or being able to directly draw in it. Keez is also thinking of adding the possibility of importing a video or a sound fragment to use as a base for your timeline. This would mean that when you are making a VR music video, you could import the song into Paperol to serve as time reference. What is already possible in Paperol now, is importing whole text files from Final Draft into Paperol. A traditional script format will be directly transformed into a Paperol format, based on a timeline. In the current version of Paperol, text that describes what a character is doing will not yet be recognized as something that will take time on this timeline. Therefore Paperol would have to learn to understand what kind of text (except dialogue) takes time and which kind of text doesn’t.
The improvement of Vaperol that will be focused on right now, will be getting a timeline in there to work. This would mean being able to put text around you in the whole VR space and making that appear and disappear at certain moments in time. Other things that Keez is thinking about adding are arrows and different text colors.
Based on some of the ideas of Paperol, Keez has also made a different tool, named Dialogus. This will be discussed in another blog of ours, to be found here.
Sharing Paperol online
Since we are sharing all of this information online anyways, we would also like to give the opportunity to our readers to try out Paperol. We hope this will give us more use cases and therefore more insight on what should be developed in the future. To be able to share Paperol with you in an easy way, PIPS:lab’s intern Sjoerd made a web-base version:
We hereby want to invite all of you reading this blog to try out Paperol and to let us know what you think. In what way will you use it? How do you make your 2D representation of a 3D space, that also has the dimension of time to take in account? Is it useful for your VR project? Why and how? Did you miss a certain function? Are you excited to see what your Paperol file could look like in Paperol VR, or do you think the VR version will not be useful to you at all? How would you want to use it? Please let us know and share your Paperol-files with us so we can see how you used our tool by sending an e-mail to: keez at pipslab dot nl. We will not share them online or use your work in any other way if you don’t want to. However, if you are ok with sharing it online for everyone to see, please let us know as well or share your work in the comments below. Small reactions within the Medium comments on the use of the tool are also highly appreciated.
This blog forms part of the Scenario and VR research trajectory, a collaboration between the Netherlands Film Academy in Amsterdam (AHK), Amsterdam University of Applied Sciences (HvA), and PIPS:lab, an Amsterdam-based collective creating multimedia installations, performances, and inventions. The research sprouts from the 360° VR movie Anyways (PIPS:lab, 2017) and includes audience research, design and development of two interactive scenario writing tools Dialogus and Paperol, two use cases regarding Paperol, and three workshops with scenario students of the Film Academy to test Dialogus. The blog series documents this research trajectory. The research is supported by RAAK.
