How to succeed in a challenging development environment

Maxim Shoustin
AppsFlyer Engineering
7 min readAug 30, 2018

“Stress is a condition or feeling experienced when a person perceives that demands exceed the personal and social resources the individual is able to mobilize.” — Richard S. Lazarus

Intro

How often did you feel rushed or panic-stricken in your development career because you messed something up in production code? How frequently has stress slowed down your productivity? We experience stress almost all time when we come across unexpected project behavior — especially in production.

There are no doubts about the importance of producing and maintaining high standards of software quality. Those standards are especially critical as we move software through the different phases of development. As an example, the likely familiar diagram below demonstrates the rising costs of bug fixes through the different phases, a heavy consequence to bear for less-than-optimal production.

With the costs and implications of “failure” heavy, what should we think about a single developer’s internal world and the consequences faced there? Among those particularly pressing are his feelings, fears, and how all will impact personal growth. It might happen that, after multiple failures have frustrated your goals, you have already put an end to dreams of a career or promotion — who knows? Consider that, from dozens of candidates, your company chose you as the best one for your job.

Keep in mind that failure is a normal process. After all, it’s a necessary part of the journey to achieving success — just perhaps a little rockier than you expected it to be. The questions then are: What should be done to minimize the risks in which you can fall prey to pitfalls or experience setbacks? Do we have tools to avoid these?

The Problem

It is worth highlighting here that, sometimes, failure is not directly related to you as an individual, but rather the result of the system and/or work environment in which you’ve been tasked to complete the “risky” steps of a work process. Consider, also, the fact that every developer has a different level of self-confidence based on the experiences he brings with him to a new company. Given that some developers still prioritize estimated time of task completion rather than program quality and stability, and are reinforced to do so through some team cultures, the possibility of error on more sensitive projects, like the AppsFlyer SDK, is greater.

Engineering our culture, as much as our products, since 1824. >>View Open Roles

For this reason, companies should be delivering clear standards and the main principles for every developer involved in the development process. An approach based on robust and stable code, rather than pressure and hard deadlines, will create a solid foundation that will, in turn, enhance developers’ chances to succeed.

Suppose that we want to publish a mobile module or SDK on the Google Play Store, App Store or with another one of our partners — Maven, Cocoapods, Nuget, etc. Imagine that it’s your responsibility to make the final publishing decision. This is a great scenario if we have only a few thousand users and the application is free.

However, what if your software is used by tens of thousands of apps — and that these apps could publish your code to billions of users? How can you maintain a positive mindset when you deal with sensitive code that could potentially damage so many applications?

Imagine what could happen if, for some reason, you broke some piece of logic in an SDK. In the span of only two weeks, you could have a minimum of about 400 applications with the latest SDK version. In the case of error on your part, you will then need to meet with each client and ask them to deploy the new application version with the new SDK, because of your mistake. Somehow, you need to ensure that such a mistake will never happen again. The aftermath from that problem can linger for a long time, maybe even for months.

Some Solutions

Don’t bury your failures — let them inspire you

Think about your failures in a positive way, and use them to inspire alternative strategies. Share your failure with your team members as a story — perhaps you host a lesson-learned meeting with your team to illustrate your persistent determination in the face of disappointment or scalable consequences.

This display of leadership and transparency demonstrates full understanding of the problems faced on the job, and highlights the best and worst practices for each troubleshooting scenario. Speaking from experience, these types of meetings are often really appreciated. Chances are, also, that someone from your audience will find it helpful.

“One for all and all for one” — Collaboration in code review

Feelings of guilt often come as a result of the human conscience. One common expression of this in the development world, for example, are feelings of intense responsibility for features that have been worked on from their beginnings. Due to the connection that may be fostered between a developer and his project over the course of its creation, feelings of guilt may also arise if any issues occur related to the feature.

“We are only human after all.” These words seem to appear with great frequency in development team settings, but no matter what your team leader may tell you, more likely than not, you will still feel guilty for mistakes. Indeed, this is also human nature.

However, what will happen if, after finishing your code, two more developers will review it? Generally, if they approve it, they’ll sign under the code that it’s their code as well. From this point on, you should feel pretty confident because the responsibility is now distributed between among your team members.

Therefore…

Stop thinking, “Code review is for juniors” or “C’mon, it’s too small for a code review.

Start thinking, “Thank you for a great opportunity to ensure a consistently high code quality.”

Having an effective code review process is an important part of any functioning development team, no matter the skill or hierarchy of the developers on it. Besides that, the code reviews define team standards, promote openness, and serve as a good “playground” for encouraging junior developers. The bigger the feedback you assert in pull requests, the better your final decision is likely to be.

Today for the Mobile SDK Team in AppsFlyer, every code change passes strict quality control review through pull requests — even for tiny changes. Code review is not simply a mundane task, but a duty with the highest priority!

Logistically, it can take between 5 minutes to hour and therefore any task planning includes an additional buffer for deliberate review time. Generally, the lifetime of a pull/merge request is one to two days depending on 1) which branch you’re working on, 2) development flow, 3) the release candidate, and/or 4) hotfix.

Overexposed, or out-of-date, pull requests, on the other hand, can lead to unnecessary maintenance and redundant conflicts during a merge.

Lastly, while code review should be performed consistently, also remember that the switch between your daily tasks and code review can cost precious time. To avoid wasting time, you can plan out your tasks wisely, before, during, and after development. For example, keep track of where you left off or schedule code review to either start or end the day.

Brace yourself, unit tests are coming!

Unit testing plays a major role in the stability of the AppsFlyer SDK. At the time of writing, we have around 400 unit and integration tests per platform (Android and iOS). Any feature comes with unit-test coverage as a matter of course. The magic is to know how to write testable code (Interfaces, Factory pattern, etc.) rather than writing code for tests. However, when we deal with E2E testing or integration testing, the test doubles like stubs, mocks, or fakes are part of a complicated world that can leave you feeling like a fish out of water, especially when rules in unit and integration tests work a bit differently than in code for release. By good fortune this enhancement inspires and motivates the team, giving developers the confidence to say something like,

“Hey, this code works like charm, I did my homework!” and not, “ … I have a good feeling that this code is good enough.” In reality, this kind of thinking undermines assurance and leads to stressful situations where developers, after publishing their code, just cross the fingers and wait for new bugs coming from customers.

Conclusion

Don’t ruin your work happiness and productivity by worrying that something can go wrong. As a start, following the aforementioned principles, such as lesson-learned information sharing; making code review mandatory; enhancing unit-test and integration test coverage, will make the product more stable and robust and allow you, as the developer, to gain satisfaction and peace.

Today, the mobile AppsFlyer SDK runs on more then 4 billion mobile devices that makes developing the SDK code super risky. This reality has pushed us to take action following the steps described above and we, as developers, are sleeping well at nights because we simply know — our code just works!

What other ideas would you add based on your experiences? You are invited to leave any comments.

--

--