Open in app

Sign in

Write

Sign in

Creating Immersive Worlds — Syllabus

Fall 2018, Interactive Media Arts, NYU Shanghai

Christian Grewell
creating immersive worlds
9 min readAug 27, 2018

--

“The only way of discovering the limits of the possible is to venture a little way past them into the impossible” — Arthur C. Clarke

Course Introduction

This introductory course will focus on building virtual worlds and understanding how immersive interactive media can help create entirely new forms of compelling experiences and narratives. Throughout the course, students will become familiar with critical concepts such as iterative design, play testing, object-oriented programming, physically based rendering of light and sound, 3D modeling, logic and experience design. They will take a design-based approach to content creation, developing a proficiency in a variety of software tools, development methodologies and creation techniques. Students will work over 7 weeks in design teams to create interactive virtual worlds to be experienced on a variety of hardware.

Course Structure

This 7-week course is split into 3 related units and three tracks (inquiry + knowledge + practice) designed to give students a multi-disciplinary foundation of experience, theory and practical skills upon which to build their immersive experiences.

The first unit focuses on human desire and progress towards a total immersive experience through innovative immersive mass media spanning the spoken and written word, theater, film, theme parks and video games. The second unit is devoted to storytelling and design at the service of immersion, including key concepts involved in modern immersive media content development via an introduction to the Unreal Engine, an open-source platform for authoring games and applications. The third unit is a hands-on series of iterative design-build-test sprints where conceptual designs are brought to life through game engines, 3D modelling, sound design and interactivity.

Course Objectives:

Students will:

  • develop digital game experiences that challenge and extend traditional immersive media assumptions;
  • professionally document their projects for inclusion in a portfolio;
  • conceptualize and prototype complex technology systems involving hardware and software at the forefront of today’s technology frontier;
  • author content that illustrates a multi-disciplinary approach to technology beyond implementation, including implications for the humanities, policy and social sciences.

Learning Outcomes:

Through lectures, media and project work, students will learn:

  • how to manage complex software and design projects involving multiple contributors;
  • about the history and current state of immersive storytelling with a focus on current and potential applications of games as tools for learning;
  • the state-of-the-art in immersive media including rendering of light and spatial sound;
  • the importance of creativity-by-combination and how to design complex and powerful game prototypes within a broader thematic structure;
  • how to juggle the requirements of authoring and publishing work to a wide audience across multiple technology platforms;
  • How to author experiences at a pre-professional level using the Unreal Engine;

Course Requirements:

  • Programming Experience: This course does not require advanced programming experience, but basic experience with 3D graphics and game engines will be very helpful. We will use prototyping techniques such as appropriation of pre-made 3D models in order to streamline the content creation process wherever possible.
  • Technology Requirements: One of the primary learning outcomes of the course is to teach you how to juggle the requirements of authoring and publishing your work to a wide audience across multiple platforms. This will require you to set up and manage a localized development pipeline and environment. Maintenance of your development environment is critical to success in the course.
  • Device Requirements: We will be publishing content for high-end PC hardware. A PC or Mac is also required to complete the majority of the course assignments. IMA has 6 PCs available in the IMA quad on the 9th floor that are open to students in the course. The instructor can also provide access to a VR lab and PCs for those not able to complete the assignments on their personal computer.

Course Resources

  • IMA Emerging Media Lab: Students enrolled in the course have priority seating in the IMA emerging media lab. This lab currently contains 6 powerful PCs with the course development environment configured, each PC has a virtual reality headset.
  • PCI Mixed Reality Studio (OASIS): The PCI Mixed Reality Studio is a new purpose built workshop for augmented and virtual reality development and production. The lab contains a green screen 3D compositing system, Optitrack motion capture camera system, Perception Neuron body tracking suit, HTC Vive and a number of powerful development PCs. This lab will be available for students during the course after appropriate training.

Books:

[Optional] Cline, E. (2017). Ready Player One. New York: Random House, Available on Kindle or Paperback.

[Optional] Herbert, Frank. Dune. Penguin Publishing Group. Kindle Edition.

Class Participation

Class participation is essential. Students will be required to demonstrate knowledge of the readings and be able to offer a critical assessment of the contents.

At times, students will be asked to lead class discussions and others will be expected to contribute to discussion based on the topic, readings and other relevant sources of information.

Academic Integrity

It is a condition of passing this course that students read and adhere to the NYU Shanghai policy on academic integrity as described in the current NYU Shanghai Academic Bulletin.

Moses Center Statement of Disability

If you are student with a disability who is requesting accommodations, please contact New York University’s Moses Center for Students with Disabilities at 212–998–4980 or mosescsd@nyu.edu. You must be registered with CSD to receive accommodations. Information about the Moses Center can be found at www.nyu.edu/csd.

Grading

Individual Project —35%

Create an immersive game experience using concepts from the course game (Taco Treasures)

Group Project —30%

Work with your team to propose, design and create an immersive video game prototype.

Journal — 25%

Write-ups, videos and other documentation posted to the Creating Immersive Worlds medium.com publication from you and your group regarding observations, play test sessions and the changes you made as a result of your observations and analysis. Evaluated on the completeness of your product journal, with special emphasis on the quality of your updates over the quantity

Participation — 10%

Do you show up for class? Do you participate in discussions (on and offline)? Evaluated on the basis of: (a) familiarity with the course materials; (b) quality of contributions in class and online; and (c) critical and creative approaches to the issue.

Schedule of Activities

Given the nature of the course and the need for us to be flexible in order to accommodate our learning goals, I’ll be proposing the schedule of activities in 1 week blocks — this ensures we stay flexible enough to cover the essential topics needed for your projects.

Week 1 Course Introduction, Immersion and Interactive Mass Media

The first session begins with the standard course introduction and syllabus overview, followed by the first part in a series on the history of videos games in the current era. The second session will focus on games as immersive mass media, including a special focus on game design concepts such as engagement versus immersion. We will play and critically analyze games to better understand what motifs, effects and mechanics best contribute to one’s sense of presence in virtual worlds.

Read:

Play:

Taco Tutorial:

This week’s technology template is a series of hands-on UE4 quickstart development tutorials:

Homework (due before the next week):

  1. Choose a video game and play it for at least an hour (terrible homework, sorry). Make a post to the your medium.com site (be sure and let me know your medium.com name so that you can add it to our class publication) describing the game. Try and encourage the reader to immerse themselves in your experience. describe the experience, paying special attention to elements of story, design and technology that increase your sense of presence. What elements decrease it? Be very very aware of the design and aesthetics of the experience.
  2. Complete 🌮 Treasure project setup here: https://medium.com/creating-immersive-worlds/world-building-434fe73e7e2e

Week 2: What is a game?

Games differs greatly from film and other linear media. A game requires input, it may or may not have an ending, and most importantly, there are consequences for decisions. It is interactive.

In these sessions we will explore what a game is and can be, and how the non-linearity of virtual worlds adds constraints that can open up entirely new forms of narrative beyond those of traditional media.

Read:

Watch:

Play:

Taco Tutorial:

This week’s tutorials continue our project by setting up our input system, introducing UE4 Blueprints and showing you how to build output messages that help inform core game mechanics and save valuable time.

Homework (due before the next week):

  1. Design a one-button game
  2. Prototype Your One-Button Game: Take the core mechanic you identified in your one-button game and use your new knowledge of the UE4 input system and blueprints to prototype your one-button game concept.
  3. Post documentation of your prototype game on the Creating Immersive Worlds medium.com publication.
  4. Play Super Mario Brothers level 1–1 and write a post on medium.com describing in (excruciating!) detail aspects of the design and controls that you feel positively contribute to the player experience.

Week 3: Game Lab

The first session continues our look into the history of video games with a focus on the new millennium. The second session begins with each student presenting their one-button game design proposal for feedback.

Read:

Play (pick one):

  • Ico
  • Shadow of the Colossus
  • Team Fortress 2
  • Half-Life

Taco Tutorial:

We introduce the camera, character and actor concepts in UE4 in this week’s tutorial series.

Homework:

  1. Group: Using the sample document example as a guide, work within your group to create a design document that includes your creative approach, theme, reference images and a variety of design techniques you would like to explore. Create a group medium.com publication, invite each member and post. Come prepared to present in the following class session.
  2. Start to prototype your game in UE4. Post documentation of your prototype game on the Creating Immersive Worlds medium.com publication.

Week 4: Design

Week 4 begins with each group presenting their game design document, followed by an analysis of good, bad and ugly level design. The Game Lab session is an interactive level design challenge where student teams will be tasked designing a game using the local environment inspiration.

Read:

Watch:

Play:

  • An indie platformer (Super Meat Boy, Cave Story, Spelunky)

Taco Tutorial:

This week’s video tutorials introduce the UE4 gameplay framework. We also begin to make our user interface.

Homework:

  1. Group: Sketch a variety of storyboards and designs for your group game and post to your medium.com publication.
  2. Post documentation of your prototype game on the Creating Immersive Worlds medium.com publication.

Week 5: Sound

Realistic sound is on of the most important contributors to your dramatic theme and immersion in your virtual world. In these sessions, we will begin with short design documentation presentations from each team, then discuss how sound design can enhance or detract from immersive experiences.

Read:

Listen:

Watch:

Taco Tutorial:

This week’s tutorial focuses on implementing core game mechanics in UE4.

Homework

  1. Create a sound ‘mood’ board for your group’s game. Include samples of UI, game play and ambient sounds. Post this to your medium.com publication.
  2. Add sound to your game prototype and post to medium.com

Week 6: Sight

In these sessions, we will explore 3D content creation at the service of immersive world design with particular attention paid to rapidly prototyping content creation at the service of realizing and testing our manifestation theme. We will explore 3D modeling pipelines, including how to model, texture and import content, as well as how to use photogrammetry and 3D scanning to create high-fidelity immersive objects and spaces.

Read:

Watch:

Taco Tutorial:

This week’s tutorials focus on AI and its implementation within the UE4 game engine.

Homework

  1. Group: Using two techniques from the tech templates, create a 3D asset for your immersive experience and add it to your UE4 project.
  2. Post documentation of your prototype game on the Creating Immersive Worlds medium.com publication.

Week 7: Play

The final week of the course will focus on documenting our immersive experiences and invite others for final play tests and feedback.

Taco Tutorial:

Below are the final tutorial series. Watch these and complete your individual project within 2 weeks.

Homework

Final documentation and downloadable build on medium.com of your individual and group game.

Note: Individual games due two weeks after the final class

Virtual Reality
Creating Immersive Worlds

--

--

Christian Grewell
creating immersive worlds

Written by Christian Grewell

221 Followers
Editor for

Hi! My name is Christian Grewell, I grew up in Portland, Oregon playing music, programming video games, building computers and drinking coffee. I live in China.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams