Reflections on Capstone and Solving Large Problems
Some Personal Reflections
This post doesn’t belong in the Capstone weekly series because, to be honest, it’s grown increasingly harder to maintain those posts as Capstone has progressed. It is safe to say that that series has been retired, since I am not able to consistently add to it.
This post does in fact contain some of my reflections about the Capstone experience.
There are three weeks left in the Capstone core program, at which point we will transition to the career search phase. In five weeks, I will be moving to NYC, where I will present on our Capstone project, meet up with fellow Capstone graduates, and apply to jobs, of course!
I could not have done this without the support that the instructors at Launch School have given me. The phrase “I couldn’t have done this without <insert someone’s name>” is used often in acknowledgements, to the point that I typically think of it as an obligatory remark rather than a sincere expression of gratitude. When I say it about the instructors at Launch School, I mean it. I literally could not have done this without them. This is not only due to the obvious reason that Capstone wouldn’t have existed otherwise, nor the fact that this program is pedagogically strong and thoughtfully crafted, but because they have truly gone above and beyond by way of providing me with a $10,000, zero-interest loan for relocation assistance. When their Capstone page says “We will invest in you first, and pour all our resources in you and align ourselves to your success,” they are not exaggerating.
The Evolution of Problem Solving: From small problems to large-scale software solutions
The project phase has been very rewarding. My team’s project has a fair bit of novelty for two reasons. First, some of the technologies we are using are themselves pretty new. Second, the ways in which we are applying other technologies are themselves relatively new. Novelty is just a perk, though. The real benefits of the project come from the engineering depth that each team tries to reach. Simplicity is really a surface-level illusion.
The project started with an absorption phase in which we collected all the research about the relevant topics and placed our focus on soaking up the material. Then, it was time to start planning and architecting. This was essentially the first step in synthesizing the massive volume of information we had just collected. With strong mental models and a plethora of information , we began outlining the core challenges we were going to solve. We identified naive and sophisticated solutions to each of these challenges and estimated the further technical requirements that these solutions entailed. We continuously worked from the outside-in, diving deeper into our system’s architecture and the algorithms that composed the system’s operations. With a high level picture of the system we were designing, we were able to dive deeper into each part without getting lost. We were able to map various milestones to each portion of our proposed design and use collaborative workflow management to track our progress.
Within each core engineering challenge we faced were many tangential challenges that we did not anticipate. This has made for an interesting experience and will make for an interesting narrative after we are finished.
We still have three weeks of full time development, so there is a lot we still need to do. I cannot help but recall my first introduction to systematic problem solving at Launch School, when we learned how to take a small problem and outline its implicit and explicit requirements; identify edge cases; map those requirements and edge cases to particular data structures and operations; and finally code up a solution. As small problems grow in complexity, they remain surprisingly amenable to the same general approach. Software development projects are obviously much larger than problem solving challenges/coding challenges, and therefore require a different way of organizing the relevant information. Fundamentally, though, larger projects just add layers to the same exact approach you would take in solving a small problem. A solid approach to solving small problems is absolutely crucial to solving larger ones, because the approach to solving large problems is an expansion of the approach you use to solve small problems.
For example, the first step in solving a small problem is to understand its requirements. In a small problem, this involves extracting explicit and implicit requirements from the problem description itself. In a software development project, this step involves a continuous cycle of research and extraction of key points, which subsequently inform the direction that the next iteration of research will take.
The next step in solving a small problem is to identify edge cases. This is no different in solving a large problem, except that edge cases become system-wide. For example, if you’re designing a distributed data-storage system, then one challenge will be: If a database server fails, how will you ensure that the data is not permanently lost? You will need some sort of data redundancy scheme. Too much data redundancy can be a bad thing because it costs storage capacity, which costs money. It can also be a bad thing because updating a single piece of data becomes updating its redundant copies as well. No data redundancy can also be a bad thing, though, because that means a data’s availability is tied to the machines that store it: hardware failures mean data failures.
The difference between solving a larger problem (with an entire system) and solving a smaller problem is, at its core, a matter of scope and scale. The difference of scope and scale is not to be underestimated. Many applications are simple to build for 100 users and very complicated to build for 100 million users. Many features are simple to implement naively and complicated to implement efficiently. A large scale solution will probably be distributed (but small scale projects may call for distribution as well), which requires choosing which distribution methods to use. This in turn requires understanding the application users’ needs and the application’s data needs. You need to know the ratios of read and write operations, the importance of things like data availability (can it be accessed at any time), data consistency (is it important that all users see the most up-to-date information), and partition-tolerance (are parts of your system likely to fail, and if so, which will be sacrificed: data consistency or data availability?) to the specific product you’re working on. As you can see, solving larger problems comes with many more considerations and many more layers of abstraction, which the problem-solver needs to continuously traverse throughout the development process.
This all sounds very theoretical but, as engineers, theory informs our decision-making. In turn, our decision-making (and the consequences of our decisions) solidifies our understanding of theory. The Capstone project is a place where all of this theory comes together. It’s where we have the opportunity to prove that all that theory was not for naught.
I look forward to revealing the project and the unique engineering challenges we faced (and conquered). Most of all, I look forward to continuously building on the solid bedrock of knowledge and practical skill with which Launch School and Capstone have armed me.
