The making of “Margaret by Moonlight”
A deep-dive into the making of a mile-wide moon-powered display.
Editor’s Note: This post is a behind-the-scenes look at Google’s tribute to Margaret Hamilton, released July 18, 2019. It’s authored by Mathew Ray, technology lead, and Isaac Blankensmith, designer, Google Creative Lab.
What does a solar power plant do after the sun goes down?
Ivanpah Solar Power Facility is the United States’ largest solar plant. 300,000 mirrors covering 4.2 square miles in the Mojave desert. During the day, the mirrors track the sun and reflect sunlight at boilers on three solar power towers. At night, the mirrors go into rest mode, perpendicular to the ground. They stay there until sunrise.
But what if they didn’t? What if they were pointed at the moon, to reflect moonlight? Could they become like pixels in a display, and create an image? And if they could, what would that image be?
This is the story of ‘Margaret by Moonlight’, a Central-Park-sized tribute to Apollo 11 programmer Margaret Hamilton. Its cast of characters include a designer (Isaac), an amateur astrophotographer (Mathew), a crew of willing co-conspirators at Google Creative Lab (where Isaac and Mathew work) and NRG and BrightSource (which run Ivanpah). It’s a story involving trigonometry, Google Earth, a moon-landing icon, and snakes. Yes, really. Let us tell you about it.
A tribute deserving of a massive platform
We first learned about the Ivanpah facility in 2017. Looking at it on Google Earth we were astonished by its size; it made the surrounding mountains seem small.
Which prompted one of the most important questions in this project: What’s the right story to tell on such a large platform?
A colleague at Google Creative Lab, Lauren, had been brainstorming ideas to celebrate the 50th anniversary of the Apollo 11 mission. Her research pointed her to Margaret Hamilton, who led the team developing the in-flight software that landed humans on the moon.
“The in-flight software itself had to be ultra-reliable,” Margaret told us when we spoke with her. “And it had to work not just perfectly — it had to work the first time. You couldn’t go up there and then try again.” There were no do-overs.
With her emphasis on failsafe software design, Margaret became a pioneer in the field of software reliability engineering — a core discipline at Google. And as if that wasn’t enough, she also coined the term “software engineer.”
“We didn’t go get Ph.D.s in it because it didn’t exist, right?” Margaret said. “But the thing is, when we were running simulations, it became apparent that we were doing as much engineering as any of the hardware engineers. So the term ‘engineer’ had meaning.”
This seemed like the perfect story to tell on one of the largest displays ever created. Margaret’s contributions to both Apollo and software engineering were epic. We’d use the light of the moon itself to help render an homage to her at epic scale.
But there was another important question we needed to answer — would we actually be able to do it?
Is this even possible?
For feasibility, we turned to our partners at NRG and BrightSource, which run Ivanpah. There were several challenges to figure out. The facility was designed to track the sun — could it also track the moon? How would an image made of rows and columns of pixels map to the radial grid of the mirrors? And where would we put the camera to capture the image made by the mirrors?
A team of BrightSource engineers in Jerusalem figured out an ingenious way of pointing their mirrors at our camera: within the software that pointed the mirrors at the power-generating towers, they created a virtual “tower” that allowed them to point the camera at an arbitrary location set by latitude, longitude, and altitude. Within a few weeks, they also implemented lunar tracking and the ability to map pixels to individual heliostats.
All they needed now was an image.
Designing the image
A traditional digital image is constructed out of a grid of square pixels aligned edge to edge. Unlike conventional pixels, the mirrors at Ivanpah are loosely aligned to a radial grid. The spacing between them increases the further they get from the central tower. To get an image to read well we’d have to design a custom method to translate our digital image to display on the array.
BrightSource was able to provide us with the exact location of each heliostat on Ivanpah 1 — with that data we were able to create a scale 3D model in Cinema 4D to simulate how images would look when displayed on the actual heliostats. A test image could be mapped to the model of the solar field and corresponding planes would rotate to on or off positions depending on the pixel values from the image at each location. This simulation gave us an idea of the resolution and general layout that we’d be able to achieve on the array and taught us a few things:
- The final composition of the image would have to take access roads, streetlights, the central tower, and mirror density into account; we didn’t want anything obscuring important parts of the image.
- Our early test renderings darkened the surrounding landscape to simulate how much light we’d have in the desert at night. We quickly realized that without the surrounding mountains, no one would have any idea how big the image was; a mile-wide image only looks like it’s a mile-wide when there is a bit of visual context.
- 107,000 mirrors sounds like a lot, but the reality is that a standard high-definition screen is over 2 million pixels. To get an image to read well on Ivanpah we had to choose imagery that was bold enough to read at a low resolution. Since the mirrors would either be reflecting the moon or not, the image had to be solid black and white. The only way to approximate any greyscale was to dither the individual pixels so they would be perceived as a gradient from a distance. We tried many methods, including a custom algorithm that took the mirror locations into consideration, but in the end we had to manually tweak the design to create the most legible image across the mirrors.
With just two weeks to the moon-landing we raced to create an image that celebrated Margaret and her achievements during Apollo. After hundreds of iterations we landed on the idea of using the array to create an image reminiscent of a mission patch. We created an illustration of Margaret based on a photograph or her taken during the Apollo days, a diagram of the moon lander and big typography.
We next had to make sure that we had a consistent way to tell BrightSource’s software which heliostats we wanted reflecting moonlight. To do this we collaborated to create a map of the solar facility with each mirror represented as a white or black dot (on or off). Our system could output the map and BrightSource could then input it into theirs to read which mirrors should be tracking the moon. This new system made it possible for us to quickly iterate on designs without having to manually update the positions of each mirror.
Planning the shoot
With the design of the image underway, we needed to know where (and when) to place a camera in order to capture both the image on the heliostats and the surrounding mountains for context.
The full moon in July happened to be arriving just days before the 50th anniversary of the landing on July 20. That posed a challenge to the timeline: We needed a bright moon, but we also needed enough time to finish everything.
That left a 2-day window: Sunday, July 15, to Monday, July 16. We’d need to get the flight paths and camera positions right the first time. No do overs.
Our first instinct was to create software from scratch to map flight paths and camera positions. But as we started to outline the requirements, we realized one was already available: Google Earth Studio.
Earth Studio was used to create accurate flight animations based on Google Earth and Maps data, complete with latitude, longitude and altitude of camera positions. It let our entire team see our flight paths far ahead of the actual shoot. The tool even enabled us to virtually rehearse and even export lat/long coordinates for our pilots to use in their navigation systems.
Another question was: how distant would the camera need to be from the heliostats for them to appear fully illuminated? The full moon is about 0.5 degrees across, and if a mirror is held at arm’s-length, the moon would be a small dot. But if the mirror is held twenty feet away, the moon would take up most of the mirror. After some trig, we figured out that our camera would need to be 802 meters from the 7-meter-tall heliostats so they’d be fully lit.
While it was important for the mirrors to be illuminated, we also needed some sunlight illuminating the landscape and mountains in the distance to get the true sense of scale of Ivanpah and how this display was created. This limited our window of opportunity to the time immediately after sunset or before sunrise, while the moon was still in the sky. We’d have only a few minutes to get the perfect shot.
Now that we had a plan for pointing the mirrors to a specific place, at a specific time, and at a specific distance away, we got to wondering — would all that light be too much of a good thing?
Since the full moon is a maximum of 0.2 lux from mid-latitudes, even if we multiplied that by 100,000 mirrors, we would still only be at 20,000 lux — about the same as a bright spotlight or digital cinema projector. That light is also spread out a bit, since every mirror isn’t 100% accurate to the millimeter. So, thankfully, our equipment would be safe from the hundred thousand moonbeams saturating them.
Countdown to launch
While all this was happening, a few other things were, too:
- We met Lori, the biologist who would ensure we didn’t harm any of the endangered tortoises who live in Ivanpah.
- We coordinated our timeline with the power plant, so we wouldn’t interfere with normal operations.
- We found an aerial film crew that could help us get the overhead shots we needed.
- We got permission from Margaret to display her face on the giant heliostat field.
- We practiced time-lapse photography with the equipment we were going to take to the desert.
- We double-checked our calculations and flight paths three times a day to make sure we weren’t making any mistakes.
- We stocked up on sunscreen and water to brave the 114-degree heat.
- We bought steel-toed boots and snake gaiters to keep us safe from both heavy machinery and the most venomous rattlesnakes in North America.
Getting the shot
And before we knew it, the moment was upon us. We all donned our hard-hats, snake gaiters, gloves, and eye protection and headed into the Mojave.
One part of the process we wanted to capture is the process of the heliostats moving into position to capture the moon. Seeing a field of mirrors the size of Central Park moving simultaneously is something few people have had the opportunity to see and we wanted to make sure we were able to help viewers understand the scale of the Ivanpah facility. We had two DSLR cameras, two tripods and an assortment of lenses: 11–24mm, 24mm prime, 70–200mm, 100–400mm and a 2x tele-extender. Overall, it was about 70 pounds of cameras, tripods, lenses, batteries, intervalometers and miscellaneous equipment in one very full backpack. One camera was used for a wide-angle timelapse of the moonrise over several hours and the other was a roving camera that was used both in the tower as well as on the ground to capture the heliostats. We settled on a capture rate of one frame every one or two seconds, depending on the speed of the motion we were capturing. If we captured thirty frames every minute that gave us a nice ratio of one frame of video for every minute of time passing.
While the time-lapse images were useful to understand the environment, we would need some airborne cameras to capture the full effect of the portrait across the expanse of the facility. Drones proved useful for lower altitude, higher precision, reproducible shots but they are not permitted to fly at the altitudes needed for our widest shot. For that we would need an actual helicopter with a director to help plan the shots, a flight coordinator to translate the shot direction into pilot-speak and a pilot to help get the aircraft into the proper position.
We met with our pilot before the shoot to go over how we’d be capturing the image. We were able to take the exact longitude and latitude that we’d found in Google Earth Studio and pre-program them into the helicopter GPS. Once in the air we’d use both the GPS and the helicopter’s built-in barometric altimeter to get an accurate reading of altitude.
We had a narrow time window of about fifteen minutes when the moon was high enough to illuminate the mirrors and the sun was still illuminating the sky to give the solar field context.
Once we were in the air, the biggest challenge was getting the helicopter to hit its mark 2000 feet above the ground with a focal point only ~25 feet wide. High tail winds overpowered the helicopter forcing us to make big loops through the focal point. Fly just a little bit too close or far from the field and the image would vignette, fly too far to one side and you’d completely lose the opposite side of the image.
We’d already calculated that the moonlight wouldn’t be bright enough to harm our camera equipment, but we hadn’t fully taken into account how hard it would be to expose a photograph that had the bright reflection of the moon and the low-evening light. Our highlights were getting completely blown out. We ended up having to set the camera to underexpose for the evening light by about three-stops to make sure that array didn’t appear as a giant patch of white.
Looking out of the helicopter window we noticed that there was a strange glowing light floating thousands of feet above the desert. We realized in awe that the light we were seeing was the focal point of the thousands of mirrors. If we could just get the helicopter to fly through that point in the sky, we’d be able to capture our image. Five minutes before we completely lost our context light, our helicopter swooped through the focal point, and were able to capture Margaret, by Moonlight.
One of us (Mathew) got lucky and had one of the best vantage points — on the tower’s observation deck 350 feet above the solar field taking time-lapse photos. From that surreal vantage point, I thought a bit about the wild mission we were on and how this strange project paled in comparison to what Margaret was doing fifty years ago.
She was entrusted with the lives of the three Americans who were to first set foot on the Moon, and she took it upon herself to go above and beyond the requirements of her team, creating new standards of rigor in software development and defining the field of Software Engineering in the process.
When we talked to Margaret, she wanted to make sure people knew that her legacy is not yet fully realized — the process of creating mission-critical, failure-proof software is a worthy cause and something the world needs now more than ever.
We only hope this tribute can not only spread the knowledge of what she accomplished, and how much further we have to go — but also what is possible when grit and determination meet a crazy idea, like sending the first human beings to the moon.
