A Career Retrospective—10 years working in tech

I’ve seen a few CS students fearful about the industry they’ll enter into when they graduate. And with all the recent tech news, who can blame them? Why am I even still here? This is my career retrospective — what has been great, what has been horrible, why I’m still here & fighting.

This was me growing up. I really loved Math and Science.

And I really loved Art. I spent half of my time on the computer, trying to write little games and make websites, and half of my time drawing on every piece of paper i could get my hands on.

And when it came time to decide what to study and what to do with my life, I felt forced to choose between the two. When I was looking into applying to art schools, the art schools I was interested in had no math classes higher than college algebra, and meanwhile I was set on learning both life drawing and multivariable calculus.

I ended up going to engineering school at MIT.

And it turns out when you’re really interested in art and engineering, the notes you take in machine learning class end up looking like this.

The first thing I did after I finished college and grad school was move to Japan and do machine learning research at Honda Research Institute right outside of Tokyo. I had taken 5 semesters of Japanese language in college, and wanted to use it before I forgot it all. I had absolutely no idea what i wanted to do with my life but I was 100% certain that robots were awesome.

The robot Honda works on is a 4 foot tall humanoid robot named ASIMO who is really adorable.

While at Honda, I worked on a reinforcement learning algorithm to teach ASIMO tasks. You might already know what reinforcement learning is but I’ll summarize it in case. Reinforcement learning is an area of machine learning inspired by behavioral psychology.

Most things that we’d like an AI or robot to do involve a series of decisions. For example, a computer AI playing pong is constantly deciding whether to move its paddle up or down or stay stationary. The inputs that go into that decision are the location of the paddles (yours and your opponent’s) and the ball.

Any time that the computer scores, we give it a reward of +1, and any time that computer loses, we give the computer a score of -1. These are the reinforcement rewards, and why it’s called reinforcement learning. And we want to build a decision-making system that maximizes rewards.

We have the computer play the game over and over. And each time the computer has to choose an action, it’ll choose based on a probability model for the current state of the game. The model takes in the current state of the game as input and outputs a decision. The probability model always starts out with all options being equal probability. At the very beginning, the AI will basically roll a 3 sided die, where 1 of the sides says “go down”, 1 side says “stay” and 1 say “go up”. As the computer wins and loses, it’ll increase the probabilities of each action it took when it won, and decrease the probabilities for each action it took when it lost. So during the next game, it’s more likely to take any actions that previously resulted in a win.

The maybe surprising thing about this method is even though it knows when it’s won or lost a game, the computer has no idea which of its actions contributed to its winning, yet it rewards them all equally. Throughout the course of a winning game, it probably had some combination of good actions, bad actions, and neutral actions. It can win even after having made some bad decisions. Is it really ok to reward them all?

A different learning algorithm, called supervised learning, says no, that is not ok. It hand-holds the AI and goes through each decision to label specifically for the AI which of its actions were good ones during a win, to only increase the probability of those.

But it turns out that even carelessly rewarding all actions during a win and penalizing all actions in a loss like reinforcement learning does, is good enough for learning. With enough iterations, the computer goes through enough different combinations of circumstances that it can eventually figure out which of the individual actions were beneficial. And this approach is great for situations were supervised learning isn’t practical, like when you’ve sent a robot to Mars and it’s encountering brand new challenges that even you don’t know about.

And it was a really great thing to learn about this point in my career. It made me feel a lot more hopeful about myself. Reinforcement learning taught me that a machine that starts off clueless gets better at making decisions just by making lots and lots of decisions. So, even though I had no idea what I was doing in terms of my career (or anything else really) I was confident that just by trying lots of different things, I would eventually become better at making decisions that resulted in the kind of life I wanted.

The novel part of the reinforcement learning algorithm I worked on at Honda was using a face-reading SDK developed at the MIT media lab. The face reading SDK would take in video input and be able to classify the expression of the person into happy, sad, frustrated, surprised, and a bunch of different emotions. It was initially developed as software to help children with autism be able to learn to read other people’s emotions.

So using the SDK, and video feed collected by ASIMO, we could detect if the human working with ASIMO on a task was pleased or frustrated with the robot. And that, instead of the concept of winning or losing, was what would be used as the reinforcement feedback into ASIMO learning algorithm. Up until that point I had thought of computer science as something really logical, and I loved the premise that to interact well with people, machines would benefit from understanding human emotions and take them into account in their decision making.

This is me with a stuffed ASIMO that I got during my time at Honda.

I liked Japan enough to want to stay another year. So I worked at Tokyo University’s Igarashi Lab, which is their version of a Media Lab. I admired a lot of their work, and when interviewing, I was happy to be a research assistant that would help with someone else’s project. But instead, they asked me what I’d like to work on if I had my own funding. I was so surprised! I mentioned that I had always wanted to design an interface for making clothing patterns. I love sewing, and sewing is very geometric in addition to being artistic, and I’ve always felt like there must be a way that computers could make designing clothing patterns easier. So when I gave that answer, they gave me funding to conduct and lead my own research, and I was still in shock at the time that something I was interested in was considered a valid research topic.

The interface that I ended up building used a life-size sewing mannequin, and you’d draw a dress design on and around the mannequin with a motion-tracking pen. And because the pen was motion tracked, the system would detect everywhere you had drawn and generate a 3D model of a dress based on it. For example, you could cut out sleeves and a neckline and even back cutouts while you were drawing. The program would then use a flattening algorithm to turn the 3D shape into a 2D pattern that you could cut out and sew. It did this for each piece by computing a transform from 3D to 2D space with the least change in difference of area and angles as computed by least squares approximation.

So I spent about year building this, doing user studies on it (one of my proudest moments was a former editor of Vogue Australia user testing my project), and publishing a paper about it.

My favorite thing about working in this lab was all the interesting projects I’d see my coworkers doing. One day, I saw a coworker cutting open a Rilakkuma bear that he had won in a crane game and panicked a little. But he explained to me I had nothing to worry about: he was cutting open Rilakkuma to put some sensors in the bear. He was turning the bear into a controller for a massage chair. You would massage the bear in the places that you wanted to be massaged by the chair!

That is to say, if you want to work on creative projects, one of the best ways to do that is to surround yourself with people with even weirder ideas than yours!

The way I’ve written about it might make it seem like my time in Tokyo was one of the the best times of my life and career. And it was. But it was also the worst time of my life and career. I didn’t mention (and often worry about mentioning, because I’d rather people focus on the research I did and the paper I published) that a week into my project, the research advisor who had been assigned to my project had told me that he had a few ideas for me, which he could only tell me over a private dinner. He continued to refuse to give me research advice unless it was over dinner. Since this sounded suspicious for a person whose job was to advise me, and so I refused to get dinner alone with him, I never got a single piece of feedback or direction and did my entire project completely unadvised. I felt very distracted from my work to have to reject his advances every single day, as he asked me out every single day for my entire time at the lab. As my time at the lab went on, his persistence escalated to him trying to follow me home and also him trying to forcibly kiss me. I felt physically unsafe to be in the lab if we were the only ones present. I later petitioned to have his name removed from the paper I published, since he had not actually given me any advice. A coworker noticed that I seemed anxious whenever I was in the lab and asked me what was going on. Relieved to have someone to confide in, I hinted at how my advisor had been acting towards me. This coworker then responded with, “Wow you sure are full of yourself aren’t you. You must thrive on drama.”

My advisor seemed unstable enough that I was too scared to report this behavior until my year-long visiting research post was over and I had left the lab and the country. After an investigation into his behavior, he was not fired immediately, but his contract with the university was not renewed. I petitioned for the university to have mandatory anti-harassment training for research advisors and rules against advisors asking out advisees, and they agreed. All in all it was a net positive, especially compared to other people’s stories of reporting abuse, but this observation from the investigation will always haunt me: “We believe that Amy’s behavior in some way led him on to believe he had a chance.” We live in a world where you can forcefully shove a man away from you when he tries to kiss you and the conclusion will still be that you led him on.

The kinds of problems that you get to work on in research settings are really interesting and novel. But I also wanted to know what it would be like to build something that people used on a daily basis. So next, I moved to San Francisco to work as a web developer for a small startup. I loved that with such a small engineering team, I got to try out everything, from front-end architecture to database schemas to customer support. Every day I told myself that I was so grateful that I wasn’t being stalked. So it took me a while to realize that whenever the site went down, it was always assumed to be my fault. I was criticized by my manager for changes I made, only to look up the lines of code via git blame and find that they were my manager’s changes. I would give suggestions and the suggestions would only be listened to if one of the male engineers repeated it. Did I mention I was the only woman engineer? This became too much to bear on a daily basis. At my exit interview when leaving this job, I was told by the founders that I should quit tech and pursue fashion instead, as they thought it was more my forte. And sadly, by the constant daily reinforcement that I was a terrible engineer with no worthwhile ideas, I had almost started to believe it.

I decided to work at a startup that was small but still large enough to have an HR department I could complain to. I loved being on the front-end engineering team at Airbnb and being a founding member of the growth team. The most interesting problems that I worked on there were around localization and tailoring the website for different languages and cultures. Some websites can put off internationalization for later, but if you want to have homes available in Tokyo for US travelers, it was really important for to have the website available in Japanese, and as many languages as possible.

I missed getting to do weird art projects though, so I tried to work those back into my life. My friends/coworkers Matt Baker, Frank Lin, and I built a webapp that we called Legoizer that lets you upload an image in order to convert it to a lego mural. It converts the colors to the Lego color space (because bricks only come in 33 colors), tells you how many pieces you need of each color, and instructions for putting it together. It also converts into lego coordinate space, because 1:1 legos aren’t square pixels, they’re taller than they are wide. While converting to lego coordinate space, it does brick anti-aliasing.

Here’s what you get if you convert starry night into legos. It turns out legoizer works quite well with impressionist paintings.

And here’s the mural that we actually built, in progress.

Another project that I worked on, with a bunch of coworkers (shoutout to Alanna Scott, Arthur Pang, Matt Redmond, Dave Augustine, and Ben Hughes!), was hacking a knitting machine, inspired by this Nintendo ad from the 80’s. The ad was featured on the video game review website Kotaku, in an article about one of the weirdest game peripherals that never made the cut.

The ad was for a game kind of like Mario paint, but it would take the designs that you painted and knit them out into little scarves. Most commenters were like “lol, I can see why this one never made it”. But I thought it was the best idea on earth. And I wanted to make it real. I researched whether Nintendo had made any prototypes i could work with. They had made prototypes but none were available to the public. But I found out that home knitting machines were a big trend in the 80’s and 90’s and I set out to find an old one that I could hack. Since it was from the 90s, the machine used to take knitting patterns via floppy drive, and so in order to send it our own patterns, we emulated a floppy drive on a computer and sent the knitting machine 1 bit bitmaps via a USB to serial port cable. Then we put the floppy disk emulator on a server so that we could upload photos to send the knitting machine via a web interface.

Two years ago, after 3 years at Airbnb, which had been my longest time I had spent at one place, I decided to try something new again.

Growing up, I had always wished that math books had had more drawings. And not just the ones to illustrate the word problems, ones to explain concepts or ones to make analogies. And I always had the idea that when I grew up, I was going to make illustrated math and science textbooks.

And so two years ago, I had an epiphany. The epiphany was, I was a grown up, now. So, it was time to make these illustrated math and science textbooks a reality.

I decided I would write zines about computer science topics. I did a Kickstarter to raise money for it. The first day of the Kickstarter was an overwhelming day. It was fully funded within 1 hour, and the New Yorker wrote an article about how I was making computer science more accessible and inclusive with this project. And it was also my last day of work at Airbnb and so many of my coworkers backed the project and wished me well, and I spent the whole day crying basically.

So why zines, and what are zines? And why was this project important enough for me to leave my full time job?

If you haven’t heard of it before, a zine is a self-published magazine. They were big in the 90’s, part of punk and riotgrrl and DIY culture. The self-published part was important to me for a few reasons. I wanted full control over what I wrote and drew. I didn’t want a publisher telling me that no 13 year olds want to read about encryption. I didn’t want a publisher telling me that no one would take these zines seriously with cute drawings. The self-published part was also important to me because I wanted people to have really low expectations. I had always kept putting off this project because I thought I wasn’t either a good enough engineer, or a good enough artist. But, zines are normally printed at a Fedex-Kinkos, and it’s normal for them to have photocopier streaks. No one expects perfection out of a zine. So I felt like calling the project a zine gave me permission to not be perfect, which was really important for me to make this project seem less scary.

I have plans to write zines for at least one more year, covering how operating systems work, image processing, and computer languages, among other topics. I’m not really sure what I’ll do next but I’m 100% ok with not knowing. Looking back at my career in retrospect, I feel like I’ve had an untraditional career path, but it’s still always gone somewhere, despite my never knowing what I’ll do next.

I have friends who are in the same job that they started when they graduated from college. I have friends who’ve climbed the corporate ladder, from jr engineer to senior engineer to tech lead or engineering manager. And I’m so proud of them and so happy for them. And I’m happy and proud of anyone who goes that route, especially marginalized people who are paving the way for more women and people of color and people of disability and LGBTQ folks, and all intersections of those, to become managers and leaders in tech. I am also convinced that following that kind of path would make me deeply unsatisfied. And I want to tell you that’s ok if you feel the same way.

Looking back at my career path, even though it doesn’t seem like a usual career path with a clear progression, it makes sense to me. It makes me feel like every weird tangent i’ve taken in my career path, had a purpose and brought me to the point where I’m combining two of my favorite things, art and technology, and getting to do that every day, with the purpose of making computer science more accessible and inclusive.

In thinking about the unifying theme of everything that I work on or do, why I do everything that I do—I was talking to a friend on twitter about why we got into computer science in the first place. When I was in high school, I loved making things on the computer and whenever I would program something and it would work and come to life, it was a magical experience.

But it’s really easy to get burned out and lose that feeling of magic. I remember struggling in classes at school, thinking I wasn’t good enough to be in this major. I remember being told by ex’s that I was dumb and wouldn’t get into grad school. I remember being stalked and harassed when all I wanted, more than anything in the world, was to be left alone so I could do my research. I remember mentorship opportunities that disappeared because they turned out to be based in romantic attraction to me, instead of an interest in my career progression. I remember being underestimated by peers and later managers, which is statistically more likely to happen to you if you’re a marginalized minority in computer science, maybe bc of gender, disability, race, or your sexual orientation. I remembered being assumed to not be an engineer because I loved to wear dresses, I remember being told that I couldn’t be promoted because I wasn’t assertive and didn’t have leadership potential. I remember feeling that the only way I’d ever be in a position of leadership was to found my own company and appoint myself the CEO. This essay doesn’t even cover all of the sexism I’ve faced in my career, because I’ve lost track and it’s too much. This essay covers just some of the egregious examples. Behind every “shocking” story a marginalized person in tech has, there are probably 50 stories of lesser magnitude that they’ve learned to shrug off.

As a result of all of those experiences, a lot of that magic i used to feel about computers as a kid has disappeared for me. But, every project that I do that combines computers and art, every knitting machine I hack, every illustration of smiley logic circuits that I draw, every zine that I write, every thing I do, is an attempt to bring that magic back for me, and if I can help bring it back for others, too, that’s more than I could ever ask for. This is why I’m still here, fighting.

“I am angry. Gender as it functions today is a grave injustice. We should all be angry. Anger has a long history of bringing about positive change, but in addition to being angry, I’m also hopeful because I believe deeply in the ability of human beings to make and remake themselves for the better.”
― Chimamanda Ngozi Adichie