Continuous delivery for data protection by design and by default — customer satisfaction, microservices and incident response plans

Johan Sydseter
Sydseter
Published in
7 min readJun 4, 2019

In a series of articles, I will present a way of continuously delivering software while protecting the privacy of natural persons by design and by default.

A good incident response plan is about more than just being compliant, it’s about customer and employee satisfaction.

Attribution: Mahnazy Azdani

Complying with the regulation

GDPR sets requirements in regards to notifying incidents and on the restoration of availability and access to personal data. article 32.1c, 33 and 34
In order to be able to meet these demands, the processor will need to instruct it’s employees in how to handle security and privacy related incidents in a way that complies with the regulation.
Conduct regular fire drills to prepare and educate your employees on what they should be doing during security or privacy breaches. The fire drill should identify improvements like possibilities for automation or gaps in the security of the system. You need to be able to verify that your organization communicates appropriately with data controllers and supervisory authorities.

Empower your incident response team

I worked as a support operator doing both 1., 2. and 3. line support for a couple of years. I was also the acting support manager for some time. What I noticed during my time doing support was that the lead-time on incidents relied mainly on 5 variables:

  • The number of hand-overs before completion.
  • Not having a clear definition of what an incident is.
  • Lacking sufficient information about the incident.
  • Lacking clear service ownership.
  • Not enforcing concise and clear communication during the incident.

Make sure you empower your incident response team and give them the tools and mandate they need in order to handle incident responses. What does that mean?
It means that they should be allowed to involve whoever they need in the organization given the nature, context, and criticality of the incident. If that means stopping the ongoing commitment of staff in other parts of the organization, so be it. Your incident response team need ownership over the incident. It’s your incident response team that is responsible for securing the ongoing communication, resolving and closing the incidents but do not forget to loop in your customer service and support desk as well. Establish a clear line between your incident response team and your customer service desk. Use a system like statuspage.io to communicate with all relevant stakeholders and refer to your status page when you are communicating with customers, users, etc.. Make sure that your service desk knows what information is required for reporting an incident to your incident response team and that they know what an incident response is. Clear service ownership needs to be established for all systems running in production so that both system administrators and the development teams are committed to fixing incidents. The ownership can be shared, but in doing so, make sure there is a person responsible for answering the phone on behalf of the team with the service ownership of the application. Keep in mind that this requires that developers also need to answer the phone during a critical incident, so make sure the development team is on phone rotation as well. Both PagerDuty and FreshWorks has solutions for this. Which one you choose depends on what type of organization you are, but regardless of which solution, make sure anyone in the organization can be looped in if required. FreshWorks has an edge in that it is marketed as a support system, but you can use it for incident management as well.

Applying “force multiplication” to your incident response strategy

No 11 Fighter Group’s Operations Room, Uxbridge (1943) (Art.IWM ART LD 4140)

During the second world war, early tests of the British Chain Home system suggested that the slow flow of information from the CH radars and observers to the aircraft often caused them to miss their “bandits”. The solution, today known as the “Dowding system”, was to create a set of reporting chains applying filters in between to move the information from the various observation points to the pilots in their fighters. The Dowding system dramatically improved the speed and accuracy of the information that flowed to the pilots. During the second world war, the Dowding system maintained an average possibility for the pilots to intercept their target at over 75%, with several examples of 100% rates. In contrast, Luftwaffe fighters attempting to intercept raids had to randomly seek their targets and often returned home never having seen enemy aircraft. The result is what is now known as an example of “force multiplication”; RAF fighters were as effective as two or more Luftwaffe fighters, greatly offsetting, or overturning, the disparity in actual numbers. ref: https://en.wikipedia.org/wiki/Battle_of_Britain#The_Dowding_system

The example highlights the importance of having a clear communication chain with multiple filters aggregating the information from multiple sources and displaying that information coherently so that it can be interpreted by your operators. If you are maintaining microservices, having this central operation room where the operators can see what is happening to the system is mandatory in order to figure out what is going on and to take decisive action to mitigate the risk of security, privacy, and downtime incidents. This central reporting and monitoring system will enable you to apply “force multiplication” in the continued operation of your system. When maintaining microservices having such a system is not only nice to have it's mandatory and required.

Incident management and customer satisfaction

The 72 hour response time mandated by GDPR should be possible to achieve for all organizations, but in order to build trust and show excellence and service mindedness when giving support, merely complying with the regulation is a poor solution for any processor that wants to have high customer retention. In fact, it’s during an incident, more than in any other context, that a processor will affect customer satisfaction for better or worse.
If you ever have experienced e.g. getting your flight canceled, then you know what I mean. Having your flight canceled, and the way the airliner dealt with the cancelation is something you remember. Whether the hostess smiled to you or not as you were entering the plane on your last flight, is something you probably have forgotten by now.

It’s therefore critical that you streamline your incident response plan. It’s not only about merely fulfilling the GDPR requirement, but it’s also about customer and employee satisfaction. Why employee satisfaction? Everyone doing a job, that wants to do their job well, wants to feel that they are making a difference. If all you care about is the bottom line, then you will probably fail in providing the culture for giving exceptional service. After all, it’s much cheaper to offshore your support, but when considering offshoring, have you ever stopped to really consider the impact that might have on your customer service? Do you think that it will help build your customer’s trust in you and improve your customer retention and add-on sales? If the answer is no, chances are that your business won’t be here 10 years from now. Offshoring your service desk means you have to increase the number of hand-overs. Due to the physical distance offshoring creates between your service desk, management- and your technical personnel, offshoring makes it difficult to provide sufficient information about the incident quickly to your service desk. The consequence is that the communication done from your service desk operators becomes much less transparent then what it could have been otherwise. This, in turn, decreases the trust in you as a company. Many support operators at offshoring facilities get a nickname so that their name becomes easier to pronounce and remember, but if your support operator has an Indian accent and bears the name Bob, chances are you won’t be perceived as particular honest and trustworthy by your customers and users. The trust and the honesty by which you are perceived indirectly affects the bottom line. This is unfortunate for you as it is the “companies that choose to be transparent under attack … that win customer loyalty through their sincerity, honesty, and openness.” (“The Customer Support Handbook”, p.131)

From a customer service perspective, customer follow-ups are key in order to show that you sincerely care about the customer or user. What happens after you have reassigned the incidence to somebody else? Will the IT-department get back and inform the customer about what the problem was, thank them for their patience and reimburse them when necessary? Whether the customer or user “ends up feeling taken care of by the technician, you’ll only find out if you check back in. customers want you, their original ally, to follow up on such questions, not just somebody over in IT, not even if you know for a fact that the IT person is the best equipped to help.” (“Exceptional Service, Exceptional Profit: The Secrets of Building a Five-Star Customer Service Organization”, p. 32)

An incident response plan is mandatory in order to be able to react to security-related incidents in a timely manner. Eliminating hand-overs is key to maintaining a lean incident response plan. The danger is that any plan involving managing incidents tend to grow out of proportions to what it is meant to achieve as it often involves several departments within the organization. Make sure you empower your incident response team and that your customer service and support desk is sufficiently looped in.

I hope these articles have shown you a way of continuously delivering microservice while doing data protection by design and by default. There are much more I could have said, but I’ll end with this.

Protecting the privacy of your users and maintaining the security of your customers is more than just being GDPR compliant. It’s about customer retention, add-on sales, and user satisfaction. If your customers are processors or controllers, they will choose you based on your privacy and security policies not because they care about your security and privacy from day to day, no, they choose you because they know that it’s the way you treat their privacy and security and their user’s privacy and security that’s going to make a difference when shit really matters. When someone gets their identity stolen or their password exposed, that’s when it really matters that you continuously delivered data protection by design and by default.

--

--

Johan Sydseter
Sydseter

Co-leader for OWASP Cornucopia and co-creator of Cornucopia Mobile App Edition, an application security engineer, developer, architect and DevOps practitioner.