An Interview with Obie Fernandez, Author of Serverless: Patterns of Modern Application Design Using Microservices
Published Jan 06, 2016 by Len Epp
Obie Fernandez is an author, consultant, traveler and photographer. Currently the Senior Vice President of Engineering at 2U, Obie is the bestselling author of Ruby on Rails books, and he has been involved in a number of successful startups and other projects. He recently launched his latest book, Serverless: Patterns of Modern Application Design Using Microservices (Amazon Web Services Edition), on Leanpub. In this interview, Leanpub co-founder Len Epp talks with Obie about his career, his books, and the inspiration behind Serverless.
This interview was recorded on December 28, 2015.
The audio for this podcast has some blips in it and you’ll see these reflected in the occasional “…” in the transcription.
Len Epp: Hi, I’m Len Epp from Leanpub, and in this Leanpub podcast, I’ll be interviewing Obie Fernandez. Obie is a New York City and Atlanta, Georgia-based author, consultant, traveler and photographer. He has been involved in professional software development and consulting for over 20 years, and most recently he’s been involved in the startup world with a variety of projects. Obie is currently Senior Vice President of Engineering at 2U, a New York City-based platform for delivering quality online degree programs. Obie is a serial entrepreneur, and in the past he has been the CTO and co-founder of a number of companies, including Andela and Lean Startup Machine. He’s also Series Editor for Addison-Wesley’s professional Ruby series and an avid EDM DJ.
Obie is probably most famous as the bestselling author of Ruby on Rails books. He’s also the author of a number of books that have been published in various states of completion on Leanpub, including The Rails 4 Way, The Lean Enterprise, and How to Eat Nachos and Influence People. He’s also currently working on a new book called Serverless. Patterns of Modern Application Design Using Amazon Web Services, which will be launching very soon, and which we’ll be talking about later in the interview.
In this interview, we’re going to talk about Obie’s professional interests, his books, his experiences using Leanpub, and ways we can improve Leanpub for him and other authors. So, thank you Obie for being on the Leanpub podcast.
Obie Fernandez: Thank you Len.
Len: I usually like to start these interviews by asking people for their origin story. I was wondering if you could tell us how you first became interested in software development, writing, and eventually in consulting in startups?
Obie: Sure. Well, I think I’ve been really lucky, like a lot of technologists of my generation. I’m 41, almost 42, and grew up in an era where it was cool to tinker with electronics and take things apart. At school, we got taught programming as early as third grade. So I was working on an Apple 2E and learned BASIC and learned Logo, and I think that that’s a real advantage, because the concepts of programming at such a young age, it just — I think has an amazing effect. Because — I don’t know, it just feels like I can’t even remember when I started programming, right? It’s like something that’s been with me for a long time.
I did eventually start getting into commercial software development with my friend Nate. He is my same age, he’s been my best friend my whole life. He started a TV and VCR repair business at a young age — always entrepreneurial — and he had a storefront selling beepers, which were like the pager things that people wore on their belts and whatnot which are not really used anymore, but at the time they were all the rage. Being responsible for the billing, and since we were both kind of hackers and whatnot, we wrote a software package that ran on his PC and used his modem to page his customers when it was time for them to pay their bill. They’d call back and they’d hit a message that said that their account was due and whatnot. We expanded that into kind of an account management system for beepers, called Beeper Pro. Unfortunately never went anywhere. We could have been like software magnates of the beeper world, or whatever.
But the little startup that we put together with a friend of his kind of leveraged that. It didn’t pan out, mostly because I think we didn’t have the attention for it. We ended up doing some early web hosting, and I learned some Smalltalk and I learned Java. We got involved in a whole bunch of other things. I was DJ’ing at the time, so I was having friends from New York come over and record sets for me to stream in 22K WAVs at danceradio.com. A lot of interesting little projects.
Eventually, I was able to get a job at a professional business. This was back in 1995. I claimed expertise with Java, which I’ll admit now was a little more fake than the interviewer [would have gathered]. But I had basically read, Java in 21 Days, which had just come out, or something like that. I think it was my first lucky break. It was a professional job doing programming when I didn’t have a computer science degree and I didn’t really have any sort of formal credentials to get a job doing so. But I was able to turn that into a series of jobs in professional consulting and over the years have had a number of lucky breaks which — together with hard work, got me to where I am today.
Consulting in general, I think is a great place if you really want to keep your skills from stagnating. Especially if you’re good and ambitious. I did a lot of consulting, then I was at a startup for 4 years in the early 2000’s, that was good, because I rode out the dotcom bust, the original dotcom bust. And then I ended up at ThoughtWorks. ThoughtWorks, during the early to mid-2000’s was the place to be. I got to work with Martin Fowler, Neal Ford and a bunch of other notable people — Fred George, who has influenced me a lot on microservices. It’s amazing what you can do.
Actually one of the common themes throughout, especially since the early 2000’s, has been kind of constant self-promotion and blogging and that sort of thing. Maybe it’s common with some of the other Leanpub authors. But in the things that I point to when I’m coaching people or advising my friends. Like, “Hey, if you really want to get ahead, pay attention to the way that you present yourself. Pay attention to the way that you credentialize yourself online. And I’ve done that. I’ve had a blog since the early 2000’s. First I was talking about Java.
I had some, a little bit of notoriety in open source Java. I worked on some dependency injection framework stuff in open source. That was kind of my first real dip into open source. And it just set me up to be in a position to talk about Ruby on Rails in a significant way in 2005, and I was one of the loudmouths that was saying, “Hey, this gives 10x productivity gains over Java. Java sucks.” and all this stuff, and being really controversial and brash and attracting attention.
That got me my first book deal with Addison-Wesley, to write, The Rails Way. Later on that led to getting the series editorship, and then being involved in all the other great books in that series. So it all points back to always being active about wanting to share the knowledge that I’ve got. Wanting to credentialize, wanting to blog, wanting to write. I mean, the first book was mostly due to something that a lot of us had or have, which is, “I’d like to write a book someday. It’d be nice to see my name on the cover of a book.” And then afterwards it turned into like, “Wow this can be a really important fuel for building your career. Because the more that you credentialize, the more that people view you as an authority, and the easier it is to get the kinds of jobs that you want, To work with the kind of people that you want. To attract other people who are also very talented and ambitious and going places.
Len: That’s a really great story. Especially the way you emphasize your own activity when it comes to gaining notoriety and getting attention. I was wondering, did Addison-Wesley approach you, or did you approach them with the idea for your first book?
Obie: Deborah approached Curt Hibbs who was an early figure in the Ruby blogging community, and who I think wrote some of the early … that were popular. He worked at Boeing and I haven’t heard from him in years, I hope he’s okay. But Deborah Willings is an editor at Addison-Wesley, who I adore. I owe her a lot. She approached him, and then he recommended me, because he just knew me from my blog. So she walked up to me and just cold-presented me with an offer to get involved in writing a book at a conference in San Diego.
Len: I’m curious, it’s something you mentioned about starting to learn programming when you were in grade three. I think this is something we’ll probably return to later, because I know that education is something that’s been an important part of many of the projects that you’ve been involved in. I’m wondering, I think there are probably people listening who wish they’d had the opportunity to start learning programming at school at grade three. Were you at a special kind of school in some way, or was it just a kind of unique circumstance?
Obie: I don’t think it was that unique for its era. I mean it was elementary school in New Jersey in Hackensack, which is the county seat of Bergen County, which is a somewhat well-to-do county — I guess. I mean, I didn’t grow up in a well to do area, I was in a working class neighborhood and working class parents. I once saw that, that particular era, there was a population dip. I guess it was kind of like an after-the-boomers sort of thing. The schools were relatively well funded in relation to the amount [of students], because the population dips, which kind of gives you a little bit of insight into how macro trends kind of play out in people’s lives. It’s kind of amazing to think about.
Len: Yeah, yeah, that’s really interesting. One of the companies you’ve been involved with is Andela — am I pronouncing that correctly?
Obie: Yeah, Andela.
Len: Yeah, Andela. I was looking at it, and on its website it says it “integrates full time genius level remote software developers into your team”. And I remember, I read a little bit about your involvement. It says it works with Fortune 500 companies to find untapped talent from around the world. I was wondering if you could explain a little bit about what your contribution was to the company when you got there? Beause I read that you sort of changed things a little bit.
Obie: For sure, happily. I joined very early on as CTO, and brought kind of that real-world experience in running a consultancy. The business model is to find very, very intelligent — I don’t know about genius level, but certainly top five to ten percentile in terms of problem solving capability, in Africa. So we operate in Kenya and Nigeria. I’m still involved as an adviser, I’m very close to the CEO Jeremy Johnson. What we did there was to find a way to put some of these young Nigerians, and now Kenyans to work, giving them opportunities that they clearly do not have on their own. It’s very hard without access to stable power, stable internet.
And then I think, one of my significant contributions there that I’m most proud of is that we realized early on that it was wasn’t just the structure concerns, right? It’s not just safety and electricity and internet. Once you have the access it’s easy to figure out the technology on your own, there’s so much wealth of information online that you can get. I mean it’s a dirty word to authors and media in general, but I mean, with piracy and whatever, where I can get a complete library of pretty much any classic book that you want to get in the field — it might be an outdated version or what not, but the access is there if you have the internet.
What you don’t have, and this is what I started figuring out about six months in, in a very vivid way, is a lot of “common sense”, what we would consider common sense — about business environment, how to deal with Western clients. What expectations are around creativity and problem solving? What kind of push back you can give if someone’s telling you to do something that you think is not the right thing? Or if you are not capable of doing what they’re asking you to do.
So that changed my whole pedagogical approach. I pretty much flipped it on it’s head, where we were putting a lot of emphasis on basic technology training and basic computer science concepts in the beginning. I started realizing now we need to really put a lot of emphasis on “soft skills” — communication skills, how to build trust, how to keep trust. How to learn, how to function on a team. How to apply creative problem solving. How to trust your own intellect when it’s appropriate, or lean on others. That sort of thing.
The mechanism for doing so was a very, very heavy curriculum of improv training, actually. A lot of people are familiar with comedic improv — the show, Whose Line Is It Anyway? with Drew Carey was very popular in the United States, and whatnot. What less people know about is a field called applied improv — implied, applied improvisation probably has a couple of thousand practitioners around the world. Consultants who come, a lot of times from the comedic improv background, but they do business consulting, and they go to companies and they do these improv kinds of activities, improv games. Some are very basic, but the idea is to get people to open up, come out of their shells.
In Nigeria, that was especially important, because the young people, and in particular young women, are just kind of culturally trained to be very quiet and shy, especially around any sort of authority figure or anyone that they look up to. It’s very hard to get them to come out of their shell. So we did a lot of work — five to six hours a day for 30 days, a program called “Month One”, where we went over, everything starting with the basic “yes, and” principle of improv. Which is that you — being constructive. So finding a way to build on what someone else is adding, trusting their intentions and then finding a way to build on the tearing down. All the way through these opening up exercises that I’m talking about, all the way towards things that we kind of learn naturally here in the States, even around some tough topics like sexual harassment and what’s appropriate and what’s not, things like that. And just basically not taking anything for granted.
The results have been remarkable. I think that the — our Andelan consultants that are now remote team members to about 50 companies here in the States and in Europe, they fit in very naturally. The way that other remote team members would fit in. It’s not the experience that people may be used to with working with offshore teams.
Len: And how does Andela find people?
The idea there is not so much training, as to see what their grit and determination is. Because you push them really, really hard, and the vast majority of them don’t have real programming experience that they’re bringing to the table. So you push them really hard, and then you see who comes out the end. So out of a class of 20 in the boot camp, we may hire five to ten. But at that point, we give you a four-year contract. And you have a job paying a middle-class salary. You get subsidized food, subsidized housing. A lot of them get to live on our campus. The audacious goal is within ten years to train 100,000 of these young Africans and inject them into the global workforce. If we achieve that, or even a fraction of that, it would make a pretty big social impact in Africa. So it’s really one of those startup situations that was very exciting and rewarding, personally, not just in a monetary sense, but also in terms of purpose, that’s what we’re doing.
Len: That’s fantastic, I didn’t know that the goals were so ambitious and positive. That’s great. And that leads me to ask you about 2U where you’re Senior Vice President of Engineering now. I was wondering if you could tell me a little bit about what 2U is doing, and what your role is there?
Obie: Yeah sure. So, as I was starting to get kind of exhausted with the travel and the intensity of working with Andela, I needed to take a little bit of a break. And I got married, I moved to New York from Atlanta. I had a bunch of life changes. My daughter Taylor went to college. So 2U is kind of a sister company to Andela, a lot of the same early investors. The CEO, Jeremy Johnson, was one of the founders of 2U. So the CTO of 2U made me an offer I couldn’t refuse. To come, help, really take their engineering department to the next level over there.
2U’s a successful startup. It’s a successful New York startup story. IPO’d I think about 18 months ago. So about 8 years old. Lots of market traction, a lot of credibility. A lot of really good brands associated with us — Yale, NYU, UNC. We have a good thing going on. We have a platform, along with the services that we provide to these schools so that they can get their graduate programs online. And what’s amazing about what’s happening there, is that … of graduate education. The best graduate program is one that has some real world experience, that one has gone out and gotten some maturity and … their career and their life to the next level.
But at that point, a lot of them have lives already. They have budding careers, or maybe established careers for a lot of the executives and professionals. They may have kids. They may not be able to move and do school full time in a particular geographic region. So what we give them is the ability to just work on it from home, remotely — while still getting the full experience, in a lot of cases, with better outcomes that they would get in person, on campus.
We now have programs that have been going on for over five years, and we can start to track the outcomes, and we see people actually having better outcomes. So the challenge there for me personally, what keeps it interesting is that it is a startup that went big and has done well. And we now have a technology department with over 100 people. The vast majority of whom are really great, really talented, really energetic. And then it’s just a question, how do you harness that talent to take us to the next level? We have a certain amount of partnership programs now, but how do we increase that by multiples?
And that — scaling technology is always kind of a fun challenge. And for me, coming back to an environment where there’s bigger teams, there’s bigger coordination and orchestration of effort, where there’s more room for applying enterprise technology or if you’re looking at the big picture, looking at strategy around it — that kind of takes me back to my days at ThoughtWorks, working with bigger Fortune 100 companies and CIO’s offices and CTOs, doing some pretty interesting large scale work.
So, it’s been amazing. I wouldn’t have necessarily predicted that it’s something that I would have been super interested in. Like especially like, let’s say five, six years ago when I was running around and doing mostly kind of smaller start up stuff. But life is more interesting when you move to different kind of areas of interest, of pursuit. And in this case, these larger scale systems are starting to become really, really interesting to me.
Len: Before I move on, asking you about some of your book projects, I’d just like to go back a little bit to what you were saying about piracy. You know that at Leanpub we’re very — we have strong opinions about things like DRM, Digital Rights Management.
Len: Around ebooks. And I was just wondering if you could — because I have no idea, but what’s your opinion about DRM and ebooks?
So in terms of just kind of the bigger picture of global equality — I mean, the fact that a lot of people — I mean there’s not — you don’t necessarily have to go to Africa to see this, you know. I know that a lot of us, when we were younger, pirated things like Photoshop. Because it was too frickin’ expensive to actually pay for it… But you know what? Now, if you use it, you pay for it. It sure helps that they created a subscription model, where you can pay ten bucks a month or something like that.
I’m generally not a fan of DRM at all. I was involved in the hacking scene, BBSs and things like that early on. So I come from a hacker background. Never really been that super concerned about it. Because the market is out there. I went with a traditional publisher with Addison-Wesley, because it was just the thing that you did. And then I continued going with traditional publishers after that; Lean Enterprise is on Wiley. There’s a big professional market out there that pays for it. Safari’s great, subscription income from Safari’s great. Stuff like that. If you’re trying to credentialize, if you’re trying to get started, it’s great to go with traditional publishers. They’ll do whatever they’re going to do. I default to getting the material in people’s hands, helping them out and basically looking at it as, if people are not paying for your material, it’s probably because they can’t afford to pay for it. Or they’re just checking it out, and you’ll get it back to you somehow later.
Len: Yeah, that’s great. I mean, that’s very — I’ve got to say, that’s very consistent with our experience as well. Especially over the last year, where some of our most successful books have been by people who are providing courses online, in particular through Coursera actually. And for them, the variable pricing that you can do on Leanpub was a necessary condition, because they had a lot of students who couldn’t necessarily afford to pay for books, and they did not want to exclude them from participating. But they also had a lot of people who wanted to pay. And so, allowing the reader the choice to pay when and whether they can or not, was really crucial. That seems to be an interesting thing that people are kind of coming to terms with, with the globalized marketplace that the internet offers, and especially in education.
Obie: I think it’s a really smart move, and I can tell the listeners from experience with variable pricing. Just because you set something at a low price, doesn’t mean that people won’t pay the higher price.
Len: Moving onto your books, some of your books on Leanpub have had interesting histories, and in some ways, they’re the ideal Leanpub books. I’d like to ask you first about, The Lean Enterprise: How Corporations Can Innovate Like Startups. First, I’d just like to ask you what the book’s about and why you chose to write it?
Obie: Yeah, so my partner, Trevor Owens and I — we have a company called Javelin. I’m not actively involved with it anymore, but before that it was called The Lean Startup Machine. Tens of thousands of people did the Lean Startup Machine experience over the weekend, over the course of the last five years, and have gotten a taste of what lean startup is about. Eric Ries of course wrote the best-selling book, The Lean Startup. If you look at the way that he launches his books — everyone has a lot to learn. All of us have a lot to learn about that sort of thing, he’s certainly one of the biggest success stories for doing that sort of thing specifically with books. Seriously, look at his latest book.
We started a whole business on applying lean startup and helping entrepreneurs and want-to-be entrepreneurs to figure out how to not waste their hard-earned savings and years of their life pursuing ideas that didn’t make sense. And then over the course of the years of doing that and getting involved in building some enterprise software around it, like basically trying to set up the go-to kind of web application for running your lean startup experiments, we talked to a lot of corporations like Nordstrom, GE, etc. that do lean startup at a large scale. And we started learning a lot about how to apply lean startup. One thing led to another, and we pitched this book idea to Wiley to basically talk about how to apply lean startup, with the target audience being senior management and the C-suite. Basically, how to establish what we call “innovation colonies”, essentially taking the experience that you would get at like, let’s say a Techstars accelerator — but doing it within the context of a corporation.
So it’s a very business-heavy topic. It’s not for the average consumer or the average technology programmers, they’re not necessarily going to get a lot out of that book, other than, maybe, things that they can pass along up the chain? But we do see it as our contribution to trying to create friendlier environments in corporations for innovation and for entrepreneurship in general.
Len: On that note, how does one get around the bureaucracy in large companies where it is not necessarily explicitly, but sort of systemically hostile to lean startup philosophies? I mean, I know Nordstrom, for example, is just fantastic. I’ve had some experience with people from Nordstrom, and they’re just great about innovating and looking for new things. But we’ve all had encounters with companies that aren’t like that. And so how does one — for example if I were say in the C-suite in a company that had a foot-dragging bureaucracy, how would I go about introducing lean startup technology?
Obie: It really has to be introduced. I mean you can do it — so there’s two answers to that. You can apply lean startup as a product manager or as a general manager. Someone who’s responsible for describing the parameters of success, the more objective that you get, the more that you rely on, build-measure-learn cycles within your business — that’s how you introduce it at the grassroots. And that’s generally successful. That’s generally viewed as a good thing, and it’s something that managers and middle executives can play up as doing things the right way, let’s say. And it can have concrete business benefits.
As far as doing an end run around bureaucracy, I mean that really has to come from the C-suite executive level. The bigger corporations are all tuned into it to some degree or another. I have a friend named Alan who’s the CTO at Coca Cola. He’s been heavily, heavily involved in innovation kind of activities. And you’ve got to realize, these sorts of things take different forms. Sometimes it’s kind of — it’s skunkworks, sometimes it’s … sort of things. But where it really starts getting, I think, super interesting and what we talk about is when those companies start sharing a significant amount of equity. So when they hook … in, they stay hooked in at the level that a VC would be hooked into early stage startups. The idea is to give people the actual freedom.
The reason we call it an “innovation colony” is because we think it’s a throwback to people leaving motherlands and going on long and dangerous journeys out to the colonies to strike their fortune. Did the colonial empires benefit from that? Sure, yeah. They would occasionally get ship loads of goods and gold coins and shit going back home. Was it dangerous and they’d occasionally lose people? Yeah sure. All the time. But it’s the whole risk/reward thing. We think that big innovation cannot happen in the bosom of a big enterprise where you really can’t fail. You have that big safety net like, “Hey you tried. Move onto something else.” You really have to go out and take the kind of risks that startup entrepreneurs take.
Len: You’ve taken some of these principles, I think, and applied them to your books. You mentioned Eric Ries before. The first Leanpub book was an Eric Ries book that was from his blog, and was sort of the — in a sense, a kind of predecessor to The Lean Startup. And I know that your books in particular — so you said, The Lean Enterprise, obviously it’s a Wiley book, you said you pitched it to Wiley. But if I’m not mistaken, it was published in-progress on Leanpub before it was completed?
Obie: It absolutely was, and I’ve blogged to prospective authors saying, “Hey do that. That’s the way to do it.” Because the traditional publishers don’t see it as cannibalizing their sales. So it provides a way to get a significant amount of income. I mean if you are already a known entity and you have the ability to market your Leanpub book — as you well know, you see the actual numbers, right? You can make tens of thousands more before the book is even anywhere near a formal publisher. And then you put it in the hands of the publisher, and they put it through their own marketing channels and whatnot. You make a substantially less percentage on a royalty basis, but I think you reach a bigger audience, so it kind of equals out.
Len: So did you have the deal with Wiley for The Lean Enterprise before you started publishing it in-progress on Leanpub?
Obie: Yes. I mean we went through a very compressed cycle. The whole thing started and was in print in six months. Maybe not in print, but kind of finished in six months. So yeah, I believe we did. I mean, I already had The Rails 4 Way on Leanpub at that point. I had already negotiated that with — at least I was kind of familiar with how to do that.
Len: I’m just really curious. Obviously we were so excited to see it when The Rails 4 Way popped up on Leanpub, and when you came on board. And of course, The Lean Enterprise as well. Was that a difficult argument that you — or case that you had to make for this process? Because it’s something that we think all books should do. Publish in-progress on something like Leanpub before the book gets taken up into the machine of the large publisher. Was that–?
Obie: It didn’t, but I’m going to try to be humble and admit that I’m probably a little bit of an outlier. I mean I have a very good relationship with them. They trust me, I trust them. So I don’t know what that experience would be like for someone who didn’t already have a track record and relationships in place.
Len: Yeah, that totally makes sense. Our hope is that it becomes somewhat conventional and an understood thing, especially for someone who’s starting out and doesn’t have a profile yet. To publish, to start publishing their book in progress, and hopefully get taken up by a publisher — if the book gets traction, and if they can demonstrate that they’re a good writer.
Obie: Yeah, the biggest challenge I think is getting that traction, and actually attracting… Like if you engage with a traditional publisher, at least in the technical world… Well first of all, actually it’s not that hard to get signed to a traditional publisher these days. I feel like a lot of the editors out there are actually following blogs and things and reaching out. And, myself, I got a lot of traction on the first blog posts I did on Serverless. On Medium I had like 15,000 reads. I got a couple of different cold emails from publishers saying, “Hey, would you like to put this on our…” You know, I got reached out from No Starch and from Apress. People who I didn’t have relationships with already. And it’s like, “No, it’s okay, I’m good.” So I can’t be the only one that’s happening with. I’m guessing that they’re going out and reaching out. Because it doesn’t really cost them very much to develop an author, to sign someone.
I think that increasingly you don’t need them as much. Because there’s this great eco system of blogs, and Leanpub is certainly part of it. That can credentialize you, you can succeed and reach a wide audience without needing them. So I think that traditional publishers are going to be increasingly, in that somewhat of a difficult situation, moving forward. The same as the case of record labels and any sort of traditional content curators.
Len: On the subject of Serverless, so you’re going to be launching it within the next week or so.
Len: I was wondering if you could tell us about what it’s about?
Obie: Yes, of course. So this is rapidly becoming one of my favorite subjects. I love catching new technology waves while there’s still a chance to get really good momentum out of it. And this is certainly the case of a movement, that being microservices, which is very rapidly ascending the Gartner hype cycle. It’s just kind of starting to be on everyone’s minds. I’ve been a fan of the concept for years now. It’s certainly not something totally new. In the earlier stages of my career, I was involved with distributed computing and SOA. And one of my earliest applications that I worked on used Forte, which at the time was one of the most advanced object oriented distributed application environments. You wrote your objects and then they had this whole UI for distributing over different nodes in your network and things like that.
So I know a lot about stuff over the years, and what I see is that it’s really kind of coming together in a way that is enabled by current technology that we haven’t seen before. Lambda is really kind of at the heart of it. It’s a new product from Amazon Web Services that lets you upload functions and run them in the fabric of Amazon’s cloud computing platform, without needing to provision servers. And that is — for someone like me, that’s very, very powerful.
Over the years I’ve gotten involved in kind of countless ventures, the whole lean startup thing, like wanting to put things out there. But you want to know that they’re capable of scaling without having to scramble and lose whatever… But you don’t want to invest a whole bunch either. And there’s this notion of T approximating, with the T variable representing development time or cost. And what happens as it approaches zero. So with technology and everything going the way that it is, you’re able to throw together software in a postmodern way, cobble together third party SAS services and APIs and libraries and open source and things very rapidly. But the final missing piece is — how do you pay for it if you want something that’s capable of scaling? And I think this answers that effectively. And this applies both to startups and also at a place like 2U, where I want to build systems that scale.
But I also want to build them in a maximally modular and maintainable way. And I want my developers to have a lot of power over those environments, and being able to really leverage all the tools at their disposal. And maintaining big, monstrous monolithic applications that have been in service for years — now, you don’t know that you’re not going to break them. You know pretty much any time you touch them, no matter how good your test coverage is and whatnot, things become very brittle after they’ve been in production for a while.
Microservices supports that; what we’re advocating with the book, is just an amazing new world, right? The microservices are basically disposable. You don’t really modify them — once they’re in service, they have a long lifetime. They’re kind of a cell, until they become obsolete. When they become obsolete, because you need them to do something differently — in the best cases, you just deploy the new version alongside it. And then you can go about in a very methodical way, testing whether it does what you need it to. Whether it doesn’t introduce regressions. Whether it performs. You can start shifting traffic over to it. You can leave the legacy microserver in production to service old clients. I mean it’s just a — it’s a very, very different world than the one we’re used to.
Len: For people listening who might not be all that familiar with microservices, can you give me maybe an example that you’ve had in your experience with something like that, where it made a big difference?
Obie: Yeah. At 2U, we do a lot of transformation of files from — basically we integrate with these legacy systems at schools. Without getting into too much details, you can think about it as 20 different partners that all have their — it’s all a very similar process, but each slightly different. So do you develop a monolithic application and clone it 20 times, and then tweak each one? I mean that’s one way of going about it. Or do you spend the time engineering that monolithic application to have all the different adaptors and configurations and strategies and all the things that you need it to have in order to be configurable for each case? But then every time you deploy it you’ve got to worry that — you’ve got to test every single integration. Or do you decompose the problem into a set of microservices, each of which does what it does really, really well? And collaborate with each other over a messaging bus, which is my favorite approach.
So that one can add — one can take it, read the format, and go, “Oh yeah, I know this format.” So it translates it, puts it back on the bus. Another service goes, “Oh here’s a translated — here’s my transcript.” And it can know what to do with this. So it grabs it and it dissects it and identifies, who is the student involved? That sort of thing.
That’s sort of decomposition without getting into too much detail. The example I use in the book is this venture called Food Button, which is the easiest way to get a meal. Some of you can probably imagine being at your desk, and you’re working and you’re in the middle of something. The last thing you want to do is stop and think about what you’re going to eat. But you need to have lunch. So I’ve dreamed about this ideal time of just being able to hit a button on an app or a physical button on the desk or whatever, on my Apple Watch, and 30 to 60 minutes later, magically food appears — cool. I’m not a picky eater. If you’re a picky eater, you will not like this idea. But if you’re not a picky eater, like me, you will love this idea. Which is press a button, 36 minutes later, magically food appears. It’s like, I don’t have to pay for it, it automatically gets charged like that.
Len: That’s a really fantastic idea. I was wondering actually when you were describing some of the issues with say dealing with 20 different systems, that reminded me of a friend of mine who works in government healthcare startup land. And it reminds me of a lot of — and I know you’ve had some of these sort of scandals in the States as well, where people trying to deal with legacy government systems find it’s very, very difficult often to change things. Because there are so many legacy systems with technology from different decades. Is microservices the kind of thing that people engaged to those kinds of very, very large projects could use to solve some of those problems?
Obie: Yeah well, I think decomposition is a trend that predates microservices. And generally speaking, that’s part of the life cycle of a large project anyway, is that eventually you start breaking pieces off of it into smaller bits and pieces. It might be new versions, a functionality, or it might just be things that make sense to partition logically. If you read Microservices by Sam Newman, it’s like, basically he makes the case for never — or maybe not never — but not starting with a microservices approach from day one. Basically, starting normally with a monolith and then breaking it off. Just really have to do not so much with the approach overall, but like the details of what’s involved.
So, for instance, when you go microservices, you decentralize your data. You no longer have kind of a one repository to rule them all. One relational store behind everything. Each microservice has it’s own repository. You get into these so-called polyglot persistence situations where one microservice is backed by MySQL, another one’s backed by Mongo. Another one’s backed by Cassandra. You’ve got your user sessions, is backed by Redis, and whatnot. And you start having this data that’s fractured over different kinds of stores in different ways, different schemas. It’s no longer clear how you do a transaction. And the truth is, you probably don’t. Which means that your developers have to be familiar with the CAP theorem. You now have to understand how these new breed systems like Dynamo favor partitioned data and cluster over consistency. So it means your systems have to accommodate eventual consistency.
These are heady topics for your average developer. This starts to take them beyond their comfort. Imagine kind of like your average Rails developer going, “But I just want to write a controller and some active record”. Whatever. And I’ve had some pushback already from friends who are kind of Agile fanatics and default to YAGNI. You ain’t going to need that, right? You ain’t going to need it. And like, “What are you doing, kind of over-complicating everything?” Well yeah, I agree, I mean you can definitely start with that approach, but there are some problems that, by their very nature, right off the bat, you know that you’re going to need to scale. You know that you’re going to be able to decompose. There’s just a talent to be able to execute that. And I think that the mix of skills that you have to pick up, like the mix of skills that we discuss in the book, basically — planning for this book to be a general reference and also a really solid primer into microservices.
I called it “patterns of modern application architecture”, because I see that this is kind of the way that the future is going to be built. Your application is no longer just a simple three layer beast with a front end and it talks to a database on the other side. We’ve got the internet of things, where you got RESTful APIs for everything. You’ve got mobile clients. You got web clients. You’ve got teams that are actually smart and want to test their shit. So you have to be able to mock things. There’s a whole modern realization of what Alistair Cockburn called hexagonal architecture, and making sure that your application has ports.
And that was in a way in that I really haven’t seen before with this serverless microservice approach for you at the functional level. You deploy them independently. You pay for them in a metered way. More and more, I just think that in the future, people are going to look back at what we did with servers. And they’re going to think it’s like contacting the electric company because you need service. And they arrange with you to set up a little power plant on their premises. It’s ridiculous, right? Like you wouldn’t think it that way. But it’s very similar to what we do now. The abstraction of a server is not part of my domain. Why do I need to think about it?
Len: So is the target audience for this — I mean, is it — is it everybody who works in software?
Len: Yeah, okay. That was one of my questions.
Obie: My favorite kind of book.
Len: It sounds like it’s making an argument as well. Which is fascinating, and those are the best books, right?
Obie: Yeah, and I’m also trying to write it in an evergreen way. So my biggest agitator so far has been whether to make it specific to Amazon Web Services. But they have a million customers, they’re dominant. I’m not too afraid to go that route. And they’re really killing it when it comes to the pace of innovation there, like Lambda, API Gateway, Dynamo. The whole story that they’re putting together and the way it ties together, I think it’s an amazing platform, and it’s not one I mind aligning with.
Len: Just for any authors listening, or potential authors. I was wondering what your plans are around the launch?
Obie: Yeah, so I’ve been — I have my personal mailing list, which is in the thousands. I have a Twitter following, stuff like that. I’ve been heads down working on the book, so I honestly haven’t thought too much about the launch. Other than to reach certain people, yourself included. I’m also going to have friends and allies that are aligned with the book tweet about it. Tim Bray is high up over at Amazon Web Services. He’s a figure that a lot of us know and love. So I’m sure he’s going to support the book. Other people that I can call upon.
So in all, just kind of leverage my network. Get the word out. And then we’ll continue to work heavily on it. It’s strategic for me in various senses. I’m also writing it for my team at 2U, because I expect that it’s going to increasingly be a part of our future there. So it helps to have your different — your work and play interests aligned in that way. It’s kind of the optimal way to do it I think. If I was trying to do this well running Hashrocket or doing a monolithic Rails app, it just wouldn’t happen. Leverage your strengths, that’s my advice to the authors.
Len: Fantastic. Thanks for that. It’s funny, when one’s already done things a few times, it can often be easy to underestimate one’s strengths and what one’s learned, and advice like that is actually really, really helpful for people who are just figuring things out.
Obie: I think if you’re struggling with your first book, realize that it does get easier. That much already I can tell. This book, I kind of launched into it, and yeah I had a lot of energy, but like just knowning the workflow, knowing how to assemble a team. Have a small team of collaborators. And I’m probably going to build it. Realize that books are generally team efforts. You need competent reviewers and collaborators to give you feedback.
If you want to write a quality book, I guess that’s the disclaimer. And I’m generally very happy with the books that I personally bought on Leanpub. But there is differing levels of quality I suppose. If you go into some of the Kindle books, like I checked out some of the Kindle books on microservices, and they were terrible. Kind of the low budget books. If you want to build a quality product, you can’t just do it on your own, you need people to review it and you need people to check your — to do the editing and stuff like that. You need a cover designer. It’s definitely a team effort to get the best result.
Len: I just have one more question. Customer development is very important for us at Leanpub. If there’s one thing or maybe even more than one thing that we could do to improve Leanpub or add to it, that occurred to you in the production of this book, what would that be?
Obie: I sent a note over to Peter. The — and I think it’s coming in Markua, am I saying that right? The ability to pull in source code from a URL would be awesome. Because in this book, I’m co-developing the Food Button source code together with the book. And at first I was like, “Oh wow, if I could only do like a Git submodule”. Beause I developed a book in Leanpub Flavored Markdown with Git — if I could pull in a submodule, it’d be great, because then I could have the source code up to date. But then that’s complicated even from a personal workflow standpoint. So I was like, “Hey what actually would be good would being able to reference a GitHub URL.” Looking forward to seeing that.
Len: Great, cool. Alright, well thanks a lot for a really great talk, and thanks for being on the Leanpub podcast and for being a Leanpub author, Obie. This was great.
Obie: Thank you guys, I love you guys for real.
This interview has been edited for conciseness and clarity.
– Posted by Len Epp
Originally published at leanpub.com.