iPHone prototype/ jimabeles FLICKR CC BY-NC-ND 2.0

The Birth of the iPhone

Excerpt from ‘Jony Ive: The Genius Behind Apple’s Greatest Products’

One morning in late 2003, just before the launch of the iPod Mini, Apple’s senior vice president of design Jony Ive and his team gathered for a bi-weekly brainstorming meeting. As usual, the team assembled around the studio’s kitchen table. One of the industrial designers, Duncan Kerr, did a show-and-tell. Kerr, who joined Apple’s design team in 1999 after having spent a few years working at IDEO, had a lot of engineering experience, and he loved to tinker with new technology.

Kerr had been working with Apple’s Input Engineering group, which was exploring alternative inputs for the Mac, with the hope of doing away with the keyboard and mouse, the mainstay of computing for more than three decades. When Kerr told the group about what he’d learned, his words were greeted by some stunned expressions.

“It was amazing,” said Doug Satzger, shaking his head in disbelief. “It was a really amazing brainstorm.”

Around the table was the core industrial design group: Jony, Richard Howarth, Chris Stringer, Eugene Whang, Danny Coster, Danny De Iullis, Rico Zorkendorfer, Shin Nishibori, Bart Andre, and Satzger.

“I remember Duncan showing us how, with multitouch, you could do different things with two fingers and with three fingers,” recalled Satzger. “He showed us on-screen rotating and zooming—and I was really surprised that we could do that stuff.”

That morning was the first time the team had even heard of multitouch. Today it doesn’t seem exceptional, but back then, touch interfaces were pretty primitive. Most touch devices, such as Palm Pilots and Windows tablets, used a pen or stylus. Screens that were sensitive to fingers, not pens, like ATM screens, were restricted to single presses. There was no pinching or zooming, no swiping up and down or left and right.

Kerr explained to his colleagues that the new technology would allow people to use two or three fingers instead of just one, and that it would afford much more sophisticated interfaces than simple single-finger button presses.

Excited by Kerr’s explanation of what a sophisticated touch interface could do, the team members started to brainstorm the kinds of hardware they might build with it.

The most obvious idea was a touchscreen Mac. Instead of a keyboard and mouse, users could tap on the screen of the computer to control it. One of the designers suggested a touchscreen controller that functioned as an alternate to a keyboard and mouse, a sort of virtual keyboard with soft keys.

As Satzger remembered, “We asked, How do we take a tablet, which has been around for a while, and do something more with it? Touch is one thing, but multitouch was new. You could swipe to turn a page, as opposed to finding a button on the screen that would allow you turn the page. Instead of trying to find a button to make operations, we could turn a page just like a newspaper.”

Jony in particular had always had a deep appreciation for the tactile nature of computing; he had put handles on several of his early machines specifically to encourage touching. But here was an opportunity to make the ultimate tactile device. No more keyboard, mouse, pen, or even a click wheel—the user would touch the actual interface with his or her fingers. What could be more intimate?

The Input Engineering team had built a giant experimental system to test multitouch. It was a big capacitive display about the size of a ping pong table, with a projector suspended above it. The projector shone the Mac’s operating system onto the array, which was a mass of wires.

“This is going to change everything,” Jony told the design team after he saw it.
This is Jony Ive’s first sketch for what would become the first iPhone: the entire front face is one big, unbroken expanse of glass. CREDIT: Apple/Samsung trial.

According to Walter Isaacson’s account in his authorized biography, Steve Jobs, Jony wanted to show the system to Jobs, but he was afraid his boss would pour cold water on it since it was still raw and unpolished. Jony reasoned that he had to show the work in progress to Jobs in private, with no one else around. “Because Steve is so quick to give an opinion, I didn’t show him stuff in front of other people,” Jony said. “He might say ‘This is shit,’ and snuff the idea. I feel that ideas are very fragile, so you have to be tender when they are in development. I realized that if he pissed on this, it would be so sad because I know it was so important.”

Jony followed his instincts and showed Jobs the system in private. The gambit worked, and Jobs loved the idea. “This is the future,” said Jobs.

With Jobs’ seal of approval, Jony directed Imran Chaudhri and Bas Ording, two of Apple’s most talented software engineers, to shrink the massive capacitive array into a working tablet prototype. Within a week, they came back with a twelve-inch MacBook display hooked to a big tower Power Mac, which provided the computing power to interpret the finger gestures.

They showed Jony and the designers a demonstration with Google Maps. After bringing up Apple’s Cupertino HQ, one of them spread his fingers apart on the screen, zooming in on the campus. The designers were astonished “We could zoom in and zoom out with touch gestures onto the Apple campus!” said Satzger.

Building a finger-controlled tablet looked like a real possibility. It wouldn’t happen overnight and, thanks to market forces, another revolutionary Apple product would emerge from the pipeline first.

Model 035

Multi-touch might have been new to Jony’s design team, but it wasn’t new in academia. The origins of the technology stretched back to the ‘sixties, when researchers worked out the first crude electronics for touch-based sensors. Systems that could detect multiple touches simultaneously were invented in 1982 at the University of Toronto, and the first workable multitouch screens appeared in 1984, the same year Steve Jobs launched the Macintosh. The marketplace didn’t see multitouch products until the late ‘nineties. Among the first were a gesture-based input pad for computers and a touch-sensitive keyboard-cum-mouse, from a small Delaware company called FingerWorks.

Early in 2005, Apple quietly acquired FingerWorks and immediately pulled its products from the market. News of the buyout didn’t leak for more than a year, when the two FingerWorks founders, Wayne Westerman and John Elias, started filing new touch patents for Apple.

After Chaudhri and Ording’s crude mockup showed that a finger-controlled tablet would work, Jony’s industrial design team set about building more finished prototypes. Bart Andre, who also has a mechanical bent, and Danny Coster led the design work. One of the prototypes they created, known internally as “Model 035,” formed the basis for a patent filed on March 17, 2004.

Model 035 was a large, white tablet that looked like the lid of one of Apple’s white plastic iBooks from the time. Though it lacked a keyboard, it was based on iBook components. The 035 had no home button and a significantly thicker and wider base than would the 2010 iPad. But the two devices share rounded edges and a black bezel surrounding the screen. It ran a modified version of Mac OS X (the mobile version of the software, iOS, was still years away).

Apple’s first prototype tablet was built from parts taken from the white, plastic iBook. Called Model 035, it was designed to run Mac OS X. The mobile version of OS X — iOS — was still years away. CREDIT: Apple/Samsung trial
Compared to the iPad 2, Model 035 was big and heavy. CREDIT: Apple/Samsung trial.
Model 035 also ran hot. Look at the line of exhaust vents all the way round the screen. CREDIT: Apple/Samsung trial

While Jony’s team worked on several tablet prototypes, Apple’s executives were worrying about the iPod. It was flying high: Apple sold two million in 2003, ten million in 2004, and forty million in 2005. But it was becoming clear that the mobile phone would one day supersede the iPod. Most people were carrying around both an iPod and a cell phone. At that stage, cell phones could store a few tunes, but it was becoming clear that, sooner rather than later, someone, perhaps a competitor, would combine the two devices.

In 2005, Apple teamed up with Motorola to release an “iTunes phone” called the ROKR E1. It was a candy bar-shaped phone that could play music purchased from the iTunes Music Store. Users could load songs through iTunes and play them through an iPod-like music app. But the limitations of the phone doomed it from the start. It could hold just one hundred songs, transferring songs from a computer was slow and the interface was horrible. Jobs could barely conceal his disdain for it.

On the other hand, the Motorola ROKR phone made it apparent to all concerned that Apple needed to make its own phone. Customers wanted the experience of a full iPod on their phones, but, given Jobs’s insistence on Apple standards, another company could hardly be trusted to get it right.

Precisely how the project that had produced Model 035 got re-tracked into making the iPhone is a matter of dispute. During an appearance at the 2010 All Things Digital conference, Jobs took credit for having come up with the idea for a touch-screen phone.

“I’ll tell you a secret,” Jobs told the crowd. “It began with the tablet. I had this idea about having a glass display, a multi-touch display you could type on with your fingers. I asked our people about it. And six months later, they came back with this amazing display. And I gave it to one of our really brilliant UI guys. He got scrolling working and some other things, and I thought, ‘My God, we can build a phone with this!’ So we put the tablet aside, and we went to work on the iPhone.[i][CE1]

Others at Apple at the time have a different recollection of the beginnings of their iPhone pursuit. They say the idea came up during one of the regular executive meetings. “We all hated our phones,” recalled Scott Forstall, a software executive. “I think we had these flip phones at the time. And we were asking ourselves, ‘Could we use the technology we were doing with touch that we’d been prototyping for this tablet and could we use that same technology to build a phone, something the size that could fit in your pocket, but give it all the same power that we were looking at giving to the tablet?’[ii][CE2]

After the meeting, Jobs, Tony Fadell, Jon Rubenstein, and Phil Schiller went over to Jony’s studio to see a demo of the 035 prototype. They were impressed by Jony’s demonstration of the 035, but expressed doubts that the technology would work for a cell phone.

The crucial breakthrough was the creation of a small test app that used only part of the 035 tablet’s screen. “We built a small scrolling list,” said Forstall. “We wanted it to fit in the pocket, so we built a small corner of it as a list of contacts. And you would sit there and you’d scroll on this list of contacts, you could tap on the contact, it would slide over and show you the contact information, and you could tap on the phone number and it would say ‘Calling.’ It wasn’t calling, but it would say it was calling. And it was just amazing. And we realized that a touchscreen that was sized, that could fit into your pocket, would work perfectly as one of these phones.”

Years later, Apple attorney Harold McElhinny would describe the immense amount of work the project required. “It required an entirely new hardware system. . . . It required an entirely new user interface and that interface had to become completely intuitive.” He also said Apple took a huge leap of faith moving into a new product category. “Think about the risk. They were a successful computer company. They were a successful music company. And they were about to enter a field that was dominated by giants . . . Apple had absolutely no name in the [phone] field. No credibility.”[iii][CE3]

McElhinny also said he firmly believes that had the project gone wrong, it could have destroyed the company. To mitigate the risk, Apple’s executives hedged their bets. They would develop two phones in parallel and pit them against each other. The secret phone project was code-named Purple, shortened to just “P.” One phone project, based on the iPod nano, got the code name P1; the other phone, led by Jony, was a brand new multi-touch device based on the 035 tablet, code-named P2.

The P1 project was led by Fadell; his group had the idea to somehow graft a phone onto a current iPod. “It was actually a natural progression of taking the iPod, which we already had, and morphing it into something else,” said the former executive.

Matt Rogers, a hotshot young iPod engineer who worked for Fadell, was given the job of creating the software for the device. As an intern, Rogers had previously impressed Fadell by rewriting some complex testing software for the iPod. As usual, the research was a big secret. “Nobody in the company knew we were working on a phone,” said Rogers.[iv][CE4] It was also a lot of extra work. At the time, the iPod team was also working on a new iPod nano, a new iPod classic, and a shuffle.

This patent filing shows how an iPod phone might have worked. The iPod’s scroll wheel acted like a classic rotary phone dialer. CREDIT: USPTO
The rotary dialer on the iPod phone could even be used to compose text messages, one letter at a time. It even had emoticons. CREDIT: USPTO

After six months of effort, Fadell’s team produced a prototype iPod-plus-phone that worked, more or less. The iPod’s click wheel was used as a dialer, selecting numbers one at a time like an old rotary phone. It could make and receive calls. Scrolling through an address book and selecting a contact to call was—unsurprisingly—its best feature. Apple filed a couple of patents from their experimentation. One of them suggested that the iPod-plus-phone could create text messages with a predictive text system. Jobs, Forstall, Ording and Chaudhri, among others, were named as inventors.

But the P1 had too many limitations. Just dialing a number was a pain, and the device was too limited. It couldn’t surf the Net; it couldn’t run apps. Fadell said later that the iPod-plus-phone was a “heated topic” of discussion at Apple. The biggest problem was that it had forced the team into a design corner. Using the existing device limited their design options in a way that was not optimal to the task. “[The P1] had a little screen and this hardware wheel and we were stuck with that . . . but sometimes you have to try things in order to throw it away.[v][CE5]

After six months of work on the iPod-plus-phone P1, Jobs killed the project. “Honestly, we can do better guys,” he told the team. Fadell was loath to admit defeat. “The multi-touch approach was riskier because no one had tried it and because they weren’t sure they could fit all the necessary hardware into it,” he said. And Fadell had been skeptical of touch screens from the start, based on experience of devices like Palm Pilots, which were clunky and awkward.

“We all know this is the one we want to do” Jobs said, referring to the P2. “So let’s make it work.[vi][CE6]

Two years later, during the iPhone introduction at Macworld, Jobs jokingly flashed an image of an iPod with a rotary dial pad on its screen. This was how not to build a new phone, Jobs said, as the audience laughed. Few knew the company might well have produced just such a phone.

Introducing the iPhone in 2007, Steve Jobs joked that this was how *not* to build a phone, but Apple only discovered that for sure after building something like it.

A New Team Takes Charge

After the decision to move forward with the P2, Jony was put in charge of industrial design, Fadell of engineering, and Forstall, previously responsible for Mac OS X, was given the job of adapting the computer operating system into a brand-new operating system for the phone.

Jony’s design team worked on the iPhone without ever seeing the operating system. They initially worked with a blank screen and later, a picture of the interface with cryptic mock icons. Likewise, the software engineers never got to see the prototype hardware. “I still don’t know what the lightning icon means,” one of the designers later remarked, referring to one of the icons on the fake iOS screen.

Jony himself wasn’t left in the dark: He was kept up to speed on the latest developments in Forstall’s new operating system, and was constantly talking to Jobs and other executives. He’d give feedback and direction to the design team. Within the design studio, designer Richard Howarth was designated the design lead of the Purple project.

At the beginning, few of those involved were confident they would be able to develop a phone. “It was fundamental R&D in all directions,” said a former executive. It meant ramping up probably the most difficult project in the company’s history and, all the while, continuing to develop products like the MacBook and the iPod line. Important staffers were moved off their current projects, delaying some products and canceling others.

There were potentially dire consequences for the company if the project did not succeed. “Had it not succeeded, not only would we have had the detriment of the lack of those products shipping, we wouldn’t have had something else to fill in at the same time,” Forstall explained.

Jobs told the executives they could recruit anyone they wanted within the company to work on the project, but they absolutely could not go outside. “That was quite a challenge,” Forstall recalled. “The way I did it is I would find people who were true superstars at the company, just amazing engineers, and I would bring them to my office and I would sit them down and I would say, ‘You are a superstar in your current role. Your manager loves you. You’re going to be incredibly successful at Apple if you just stay in your current role and keep on doing what you want to do. I have another offer for you, another option. We’re starting a new project. It’s so secret, I can even tell you what that new project is’ . . . and amazingly, some tremendously talented people accepted that challenge and that’s how I put together the iPhone team.[vii][CE7]

Forstall commandeered an entire floor in one of the buildings at Apple HQ and had it locked down. “We put doors with badge readers, there were cameras, I think, to get to some of our labs, you had to badge in four times to get there,” he said. It was nicknamed the “Purple Dorm.”

“People were there all the time,” Forstall said. “They were there at night. They were there on weekends. You know, it smelled something like pizza.

“On the front door of the purple dorm, we put a sign up that said ‘Fight Club’ because the first rule of Fight Club in the movie is you don’t talk about Fight Club, and the first rule about the purple project is you do not talk about that outside of those doors.”

Designer Chris Stringer’s early sketches of the iPhone look rather like the finished product, but with a smaller, bordered screen. You can also see sketches for an extruded-aluminum model, which would be abandoned during rocky development. CREDIT: Apple/Samsung trial

Over in IDg, Jony began, as usual, with the iPhone’s story. As he later explained, it was all about how the user would feel about the device. “When we are at these early stages in design, when we’re trying to establish some of the primary goals—often we’ll talk about the story for the product—we’re talking about perception. We’re talking about how you feel about the product, not in a physical sense, but in a perceptual sense.”

Jony believed the iPhone would be all about the screen. In their earliest discussions, the designers agreed that nothing should detract from the screen, which Jony likened to an “infinity pool,” those high-end swimming pools with an invisible edge.

“What that did was make it very clear in our minds that the display was important, and we wanted to develop a product that featured and deferred to the display,” he said. “Some of our early discussions about the iPhone centered on this idea of . . . this infinity pool, this pond, where the display would sort of magically appear.[viii][CE8] The team made a point in exploring design ideas to avoid any approaches that would diminish the importance of the display.

Jony said they wanted the display to be “magical” and “surprising.” These were his high-end goals for any eventual design. “In the earliest stages of this design, this seemed—this was very new, and it felt there was real opportunity to develop a design story based on those sorts of preoccupations,” he later explained.[ix][CE9]

Late in the fall of 2004, Jony’s design team began work on two distinct design directions. One, called “Extrudo,” was led by Chris Stringer, and it resembled the iPod mini. It was made from a flattened tube of extruded aluminum and could be anodized in different colors. Apple already had big production lines making and anodizing iPod cases in huge numbers. That was one advantage of that direction, along with the fact that Jony and team loved what could be done with extrusion.

The other design, called “Sandwich,” was led by Richard Howarth. Made mostly of plastic, with a plastic screen, the Sandwich design was rectangular with evenly rounded corners. It had a metal band running around the midpoint of its body, a centered display on the front face, a menu button centered below the screen and a speaker slot centered above the screen.

The iPod Mini was an early inspiration for the iPhone. Apple sold millions of them and had big production lines already set up. The designers felt the Mini’s lines were simple, modern and clean. Flickr CC: m-s-y

Jony and his team preferred the Extrudo look and gave it the most attention. They tried cases that were extruded along the x-axis, and some along the y-axis. But problems surfaced immediately. Extrudo’s hard edges hurt the designers’ faces when they put it up to their ears. Jobs especially hated this.

This is an early CAD rendering of an extruded aluminum iPhone. Note the plastic end caps; they’d become a huge design problem. CREDIT: Apple/Samsung trial.
This is an early model of an “Extrudo” iPhone made from extruded, anodized aluminum, just like the highly-successful iPod Mini. CREDIT: Apple/Samsung trial
The designers played with extruding the aluminum case along the X- and Y-axes, but with each iteration, the plastic end-caps got bigger and bigger. CREDIT: Apple/Samsung trial

To make the hard edges softer, plastic end caps were added, which also helped with the radio antennas. The iPhone would have three radios: Wi-Fi, Bluetooth and a cell radio. But radio waves won’t pass through a metal shell, so the plastic endcaps became essential.

The early iPhone prototypes betray their iPod heritage with “iPod” etched into their backs. The screen is a sticker printed with fake icons. The designers were not allowed to see the operating system as it was being developed. CREDIT: Apple/Samsung trial.

The team struggled to solve Extrudo’s problems, but engineering tests made it clear this particular design direction wouldn’t work unless the plastic endcaps for the radios got bigger. But bigger caps would ruin the clean Extrudo look. “We made books and books filled with pages of designs trying to figure out how not to break up the design because of the antenna, how not to make the earpiece too hard and sharp, and so on,” said Satzger. “But it seemed like all the solutions that added comfort detracted from the overall design.”

The Extrudo design had another problem that nagged at Jobs: The metal bezel detracted from the screen. The design didn’t “defer” to the screen, which had been one of Jony’s original goals. Jony later recalled his flush of embarrassment when Jobs pointed it out.

Apple killed Extrudo; the team was left with Sandwich.

This is a CAD rendering of a “Sandwich” design, which envisioned the phone of a metal frame sandwiched between a plastic screen and plastic back. CREDIT: Apple/Samsung trial.
This is an early “Sandwich” prototype (also marked “iPod” on the back) which imagined the iPhone in Apple’s iconic white plastic. In early models, the Home button is marked “Menu.” CREDIT: Apple/Samsung trial.

The Sandwich design did have several advantages over Extrudo, one of which was that the rounded edges didn’t hurt the designers’ ears. But the engineering prototypes came back big and chunky, and Jony’s team struggled to slim it down. They were trying to cram in a lot of technology, much of which hadn’t yet been miniaturized enough for a device as complex as the phone everyone envisioned.

Apple’s designers struggled to fit all the components into the phone. Successive Sandwich designs got fatter and fatter. CREDIT: Apple/Samsung trial.

Reprinted by permission of Portfolio/Penguin. Excerpted from Jony Ive: The Genius Behind Apple’s Greatest Products. Copyright 2013 Leander Kahney. All rights reserved.Also available in the UK.

[i] John Paczkowski, “Apple CEO Steve Jobs live at D8,” http://allthingsd.com/20100601/steve-jobs-session/, June 1, 2010.

[ii] Scott Forstall, Apple v. Samsung trial testimony

[iii] Ibid.

[iv] Kevin Rose, Matt Rogers: Founder of Nest Labs interview, Foundation 21, video, http://www.youtube.com/watch?v=HegU77X6I2A

[v] “On the verge,” The Verge video http://www.theverge.com/2012/4/30/2987892/on-the-verge-episode-005-tony-fadell-and-chris-grant, April 29, 2012.

[vi] Walter Issacson, Steve Jobs, Kindle Edition.

[vii] Scott Forstall testimony at Apple v. Samsung trial.

[viii] Apple v. Samsung trial, deposition of Jonathan Ive,

[ix] Apple v. Samsung trial testimony. (Ibid.)

Leander Kahney