In the Realm of the Senses:
Tetsuya Mizuguchi’s Synthesis of Synaesthesia & Video Gaming
Part I: The Birth of an Interactive Artist
The 5th of December, 2015. The Moscone West exhibition hall in San Francisco is host to the 2nd annual PlayStation Experience, where video game developers and publishers the world over hype their upcoming games for PlayStation platforms. With Sony having recently announced a 2016 release date for their virtual reality headset imaginatively named PlayStation VR (previously codenamed Project Morpheus), attendees were curious as to what new gaming experiences VR would bring. With the very personal experience of virtual reality notoriously difficult to present to an audience, the initial awkward demo of the technology met with a lukewarm reception.
But then, a familiar countdown resonated through the hall…
14 years after its release on Sega’s beloved but ill-fated Dreamcast, the cult “music shooter” Rez was announced to be updated for virtual reality as Rez Infinite, running at 60 frames per second (120 for VR) in 1080p HD. Accompanying the high-resolution wireframe graphics on the stage was a man in a glowing “Synaesthesia Suit”, reminiscent of the costumes in the movie TRON.
Before the presentation even ended, previously skeptical gamers were now sold on virtual reality. When it was revealed that it was Rez’s mastermind Tetsuya Mizuguchi himself in the Synaesthesia Suit, the audience erupted into rapturous applause. For many, though, the man behind the VR headset remains a mystery.
Born on the 22nd of May 1965 in Otaru, Hokkaidō, Japan, Mizuguchi’s first experience with video games came at the age of 11 when he played the seminal 1972 tennis game Pong at a friend’s home. While gaming and baseball were among the young Mizuguchi’s interests, he was particularly affected by music:
“Music is like a universe — many, many elements. Lyrics, chords, rhythm, beats, playing, listening. Emotionally, you feel something. When I was 11 years old, I listened to my first Beatles music. I couldn’t understand English, but I felt emotion or something, sort of… I want to love somebody! It’s the power of music, of chords… very physical things, very emotional things.”
Wired, Interview: Mizuguchi and Matsuura, Music Gaming Geniuses, 12th February 2008
Choosing to study Media Aesthetics at Nihon University’s Faculty of Arts, Mizuguchi explored the relationship between passive and interactive media and began to believe in the superiority of the latter:
“I love movies, but the movie is almost complete as media,” he says. “And very domestic, too — each country has its own market, except Hollywood. Music was the same. But the game, that was worldwide, interactive. Low resolution, low technology, but I could imagine the future.”
He attended computer graphic imaging conferences in Europe and the US and perceived the true dawn of virtual reality. “I saw the games industry as a white canvas,” he says. “I thought, I can draw something.”
TheAge.com.au, Serial game player, 9th November 2004
His desire to explore and combine the various mediums of art and entertainment would stay with him throughout his life and work. Upon graduating he directed some short films in his spare time along with a few music videos for his friends’ bands, having grown up as teenager with the garish neon-filled MTV music videos of Cyndi Lauper, Duran Duran and XTC. Moving to Tokyo he found a job in an advertising agency for a television company, but did not find it creatively fulfilling:
“I tried to learn what was creative in TV and advertising, but I could feel it wasn’t my future. It was a very limited creativity. I lost my way.”
But a fateful visit to a Tokyo arcade would determine his career path.
It was here that he laid eyes upon Yu Suzuki’s G-LOC running on the gyroscopic R-360 arcade cabinet; seeing the player flying a fighter plane, upon getting shot by enemies the machine would spin on its two axes. In awe of the hardware, Mizuguchi wondered who was responsible for such an innovative gaming experience. He looked at the arcade cabinet and spotted the manufacturer’s logo: SEGA.
“I saw that, and I was like ‘wow’. I saw the logo, Sega, and thought that’s it — I want to join Sega!” Mizuguchi went straight to the company’s headquarters in Haneda. “There was no appointment,” says Mizuguchi with his crooked smile. “But I wanted to join this company. The lady at reception, she said to me, you need to have an appointment. So I said, okay, give me an appointment!
“Sega wasn’t such a big company at the time, it was very small. I got an appointment with the human resource division, then I got an interview. I forget what exactly I said, but they picked me out. I said to the director of R&D, I explained to him I don’t want to make a game. I want to make a future game. The current game, it’s too young, too childish. Games should be a new entertainment form, in the future. This is too small. This is still otaku culture. But it’s going to get huge in the future. I believe that. And I want to do that.
“Then he says ‘Oh, you don’t want to make a game? This is a game company. But I like you!’ So I joined Sega. There were no other candidates for me like Nintendo and Namco. It was straight to Sega.”
Eurogamer.net, In media Rez: the return of Tetsuya Mizuguchi, 8th February 2015
Upon joining Sega, Mizuguchi continued his research into games as media and looked at forecasts of the future of the medium, coming to the conclusion that people in the industry were eager for virtual reality.
In 1991 Sega sent Mizuguchi to the UK for talks with Virtuality; the previous year saw the fledgling company announcing that they would sell the first commercially available virtual reality technology.
The first Virtuality models, powered by an Amiga 3000, were released in October 1991.
It wasn’t long after this meeting though that Sega announced plans to develop their own virtual reality headset, and 1991 saw Sega announce the Sega VR for their Genesis/Mega Drive console.
Partnering with Reflection Technology, the Sega VR was to retail for a mere $199 and be bundled with four games.
A year after a presentation of the device at CES 1993, however, the Sega VR project was terminated.
While Sega’s official reason for cancelling the Sega VR was that it was “too realistic”, in September 2015 Tom Kalinske, who was CEO of Sega of America during from 1990 to 1996, revealed:
“It was pretty cool. You could put it on and literally you were in a virtual world, and it moved as you moved your head.
“One of the problems was almost everybody got sick. It caused severe motion sickness. Other people got severe headaches. I think we were right in turning it down.”
/r/retrogaming Podcast, Episode 07: Tom Kalinske Speaks About His Time As Sega of America’s CEO, September 2015
Though the home console version was scrapped, the technology was released to arcades in 1994 as the Sega VR-1:
Seeing their first attempt to enter the VR consumer market fail with Sega, 1995 saw Reflection Technology announce a virtual reality headset with Sega’s rival Nintendo: the infamously commercially unsuccessful (and headache-inducing) Virtual Boy.
While VR floundered, Mizuguchi set to work at Sega with a new unit tasked with exploring pre-rendered computer-generated imagery (CGI), designing a CGI film and simulator ride for Sega’s Advanced Simulator 1 (AS-1).
“That was [a tough] project, because nobody knew about CGI. ‘What is a digital movie?’ We learned a lot. At Sega there was no staff like that specialised in it. All the staff at Sega, they just wanted to make a game. But this was the first visual expression project for Sega.”
Michael Arias began his career in the entertainment industry as a camera assistant on motion control stages for films such as The Abyss and Total Recall, he worked as a motion control programmer for Universal Studios’ Back to the Future: The Ride, directed by Douglas Trumbull, famed for his groundbreaking effects work on Stanley Kubrick’s 2001: A Space Odyssey.
In 1991 Mizuguchi met Arias and Trumbull in Massachusetts and invited the two to come to Tokyo. After a few months as a motion-control camera operator at Imagica, Arias joined Mizuguchi at Sega. Together they conceived the Blade Runner-esque Megalopolice: Tokyo City Battle.
After two years of gruelling work with the small team of five coming to grips with new technology, Megalopolice was released in 1993. The same year saw the release of another AS-1 ride, an interactive game featuring the King of Pop:
The scandals surrounding Michael Jackson saw Scramble Training pulled from Sega arcade centres in 1994, but not before it made an appearance on an episode of the French Disney Club TV programme.
A trip to Sega Europe would provide inspiration for Mizuguchi’s next venture.
“I wanted to make a racing game, but I had to think about what the concept would be, and what the differences would be. I met up with people from Sega France, Sega Italy. I asked them — remember I was a new face at the time, so I didn’t know so much about the industry at first — I asked them for their inspirations and their opinions. ‘What kind of game do you want? What kind of vision do you have?’ Most of the people had complained to Sega headquarters that they only watched the American market, for example, things like Daytona. So Sega of Europe people were complaining about this, but they didn’t have the answers either.
“So I was watching a TV program, a sports channel, and I watched rally racing live. That was my first experience watching that. I said, ‘What is this?’ My original idea of rally racing was of the Paris to Dakar rally, with trucks, which is very different. But the WRC [World Rally Championship] is raced using regular cars you can buy; Toyota, Lancia. Many, many cars, cool stickers, driving into cities and forests, there are many people on the side of the road, including old people and young kids, female, male. It was like a festival, so passionate. Then it occurred to me: ‘Let’s make a rally game.’ Daytona was limited to circuits — grey circuits. But with rally we could design many landscapes; nature, cities, desert. So we decided to make a rally game. That was the start.”
The new Sega Model 2 arcade board, designed by Sega AM2’s Yu Suzuki, used hardware conceived by Lockheed Martin for texture-mapping of polygons. Mizuguchi aimed to use this technology to bring colour and vibrancy to the previously dour rally genre. A team of 12 was assembled at Sega AM3, among them the game’s director Kenji Sasaki, who had worked on the Namco arcade title Ridge Racer (which would later become a popular series on the PlayStation).
“To me it felt like a very long, long time. We were so young! I was 28. And everybody was in their 20s — 25, 24. It was everybody’s first experience to make that kind of 3D, CG game. Most of the people, it was their first time making a driving game, including me. And we wanted to make some revolutionary game! We were like, okay, let’s forget about [existing racing games] and do an original thing. What can we do with the textures we can use? Let’s forget about circuit games. Let’s go out to nature. I was into rally itself, and the World Rally Championship in Europe — real cars with many stickers that drive through the town and the forest and the desert, with many people screaming. I felt this passion, and wanted to put that passion into the game.”
Mizuguchi took his experience from his years working with CGI and together with Sasaki set to work to convince Sega of the viability of their concept:
“So I made a demo movie, which I think was the first case it had been done in Sega. I made a three-minute movie, just an image movie, compositing a real rally game, rally racing. And Kenji Sasaki, Sega Rally’s director, he could make CGI movies too, so I made a composite mixture of desert, forest, jumps, sliding turns… very exciting. Then everybody changed their expressions and said ‘Hmm.’ But there were many arguments; for example, there’s no oval. So I said ‘what do you think?’ They said ‘No, no, no, let’s make an oval rally course.’ I thought we could make many, many stages — desert, city, forest, many colors, jumps, drifting, in rocky type environments. I felt I could make a very passionate new racing game, but it was a big battle.”
This desire to have a rally game in a new exotic setting meant that the project began life as Pacific Coast Rally, with the player racing round the winding rounds of the famous California Pacific Coast Highway:
“I felt we should use that area, so we went to San Francisco with five or six people. We had cameras for designers to shoot textures. We took two weeks to tour San Luis Obispo, Death Valley and Mexico. That was fun; it was like a road movie for a road-based videogame. We always had our meetings in our car. It was an organic process discussing what kind of stages to make. We had many, many inspirations. “I had experienced that area before, but the young designers and artists didn’t know about it, so I said, ‘Let’s go.’”
But this conflicted with their objective of authenticity, as the Pacific Coast Highway was not host to car rallies and it would seem odd to have real rally cars racing outside of their normal environment. Thus, Pacific Coast Rally became Sega Rally Championship 1995.
Licensed cars became a common feature in subsequent racing games such as Gran Turismo and Forza, but at the time there was trepidation from both the accountants at Sega and the car companies themselves who saw video games as the pastime of juvenile delinquents.
“I decided to visit Toyota, but this was a new thing. I was young, with no experience, but also no fear, so I said ‘Let’s go!’ and I met with a Toyota PR manager. I was like a kid. I said, ‘I want to make a rally game using your car.’ I wanted to make a game using Toyota Celica and Lancia Delta, because they were both champion rally cars, but there was no synchronization [between the two manufacturers]. The Lancia Delta was a former champion car, and the Toyota Celica was the new champion car. But they never fought each other for the championship, so it was a fantasy setting to have two real champion cars fight in a virtual Sega rally race.
“So I went to Toyota. I brought the demo on a video tape. They hated videogames, because until then they thought videogames were a fake business. No publicity, no PR, so they really hated the videogame industry. But I went there and brought the Sega Rally demo — the first 3D demo with textures — and they were so surprised. They said, ‘This is a game?’”
After a few hours, we had a very hard discussion, and I told them if we could have some licence from you, this arcade machine is going to spread to the world. It’ll be a big promotion — and everyone will want a Celica.
“It was not so easy to get the licence. He said if Fiat — who did the Lancia Delta — if they accept, then we accept. So I go to Sega, and tell them I want to go to Italy — immediately! The head of Sega’s R&D said no. We had no insurance. If you go there and they say no, that’s it. I had to go. This is my life! Then he said, okay, let’s go. I went there and met the guy, and I explained the same thing. They were so impressed. Then I them to sign, go back to Toyota and got them to sign. That was the first game that had that kind of collaboration with real companies. I was really proud.”
The ten-month development period was as eventful as it was brief, thanks in great part on Mizuguchi’s insistence that the team try out the cars themselves:
“Many times I took them to the circuit in Japan. This was a dirt circuit, no asphalt. We borrowed real rally cars — and we got insurance, so if we hit the car that’s okay.”
Cars were hit. Numerous times. Mizuguchi himself had an altercation with a wall that left a car half-crushed, one of several vehicles sacrificed in the name of authenticity. “That kind of experience, it was really important. If you slide the car and hit the brake, what kind of movement do you have? When you feel the axle slide, it’s so much fun! Making the game, we had big arguments. What is the drift technique? What’s the correct counter steer?”
It was experiences such as this that helped the team stay on track, as Sasaki would later recall:
“To be completely honest I was very worried that the game was going to be a failure at the arcades. It was only as the game started to take shape from the team’s personal touches that I started to feel any confidence in its potential. At one key point while we were developing I had been working so hard, always with cars on my mind, that I got to the point where I just couldn’t see the attraction anymore. I’d think, ‘What the hell is so exciting about cars? What is so fun about driving?’ So I drove up into the mountains with my own car. It was such an enjoyable and exhilarating experience that I decided to incorporate this into the game. This was how the third mountain track in the game was conceived and decided on.”
Edge, The Making Of: Sega Rally Championship 1995, 2nd October 1999
The technological innovation, atmospheric environments, and attention to detail in Sega Rally, where players would have to adjust their speed and handling depending on what surface they were driving, would later become typical of Mizuguchi’s work.
Sasaki’s fears of failure were unfounded. With Sega shipping 12,000 cabinets to arcades in Japan, North America, and Europe from October 1994, Sega Rally Championship 1995’s success meant a home conversion was inevitable.
The Sega Saturn launched in Japan on the 22nd of November 1994, and upon the release of Sega Rally Championship for the console in December 1995 (January 1996 for Europe) it proved equally popular, going on to to become the Saturn’s second-best selling game with 780,000 copies sold during the console’s short life. A PC port would see home sales reach 1.2 million.
The bulk of the conversion work was done by Sega’s CS Team, with Mizuguchi and AM3 supervising. The technical inferiority of the Sega Saturn compared to the Model 2 meant that the home conversion could only manage 30 frames per second compared to the arcade’s 60, and the rushed US release of Sega Rally Championship led to last-minute bug fixes and graphical improvements being made to the Japanese and European versions. Mizuguchi also worried about the experience of playing on the Saturn controller versus the arcade cabinet’s force feedback wheel, But the Saturn did have one bonus feature that proved particularly enticing to Japanese fans, as Mizuguchi explains:
“In the Seventies, this was the monster champion car. It’s so nice. The Lancia Stratos was like a super-supercar. It was a classic, but everyone in Japan and Europe knew it. In Japan when I was a kid, there was a supercar boom, and everyone had rubber erasers shaped like cars. All the kids had many rubber cars, and everyone knew about this car, the name, the detail. So we wanted to put a big surprise in the Saturn version. I remember I put the Lancia Stratos’ name in the last spot in the credits roll. The car was like an actor appearing in the game: ‘Starring the Toyota Celica, Lancia Delta Integrale, aaaaaaand the LANCIA STRATOS.’ Drama! Everyone was like ‘What!?’ And the next time you played you could select the Lancia Stratos.”
On the 26th of September 1996, Sega Rally Championship Plus was released in Japan, allowing Saturn owners who owned an XBAND modem to race each other; the May 1997 release of Sega Rally Championship Plus: NetLink Edition allowed US Saturn owners in possession of a Sega NetLink modem to do the same.
After working with simulator machines for years at Sega, Mizuguchi had finally gotten into developing games:
“That was so nice, as a first step… I learned basic game design skills, and I watched the worldwide market. But in my mind, I wanted to make something very emotional.”
It was around this time that the seeds of an idea were planted in Mizuguchi’s mind:
“I had many inspirations for the synaesthesia. My first rave party in 1994–95, with all the techno music, lights, colours, a group of a thousand people dancing, almost moving as one with the music. That was very shocking to me… in a good way! I was reminded of that and Kandinsky’s concept of synaesthesia. I thought, ‘If I could bring this kind of feel to a game, what kind of game can I make?’ I didn’t have any answers at that time. I needed time.”
IGN, Tetsuya Mizuguchi: A Child of Technology, 15th May 2011
Mizuguchi had little time to reflect on his achievements or to leave the racing genre, though, with the next few years proving to be his most prolific.
