Continuous delivery for data protection by design and by default — Security and privacy reviews
In a series of articles, I will present a way of continuously delivering software while protecting the privacy of natural persons by design and by default.
The supervisory authority is tasked with periodically reviewing the operation of the data controller. (article.41)
Don’t let the supervisory authority catch you off guard, conduct regular reviews for evaluating the system’s security and privacy.
The regulation states that “the controller and the processor shall implement appropriate technical and organisational measures to ensure a level of security appropriate to the risk, including inter alia as appropriate:…a process for regularly testing, assessing and evaluating the effectiveness of technical and organisational measures for ensuring the security of the processing.” article 32.1d If the supervisory authority decides to audit your operations and find that your not reviewing the “effectiveness of technical and organisational measures for ensuring the security of the processing”, they may question your security and privacy practices and fine you appropriately. To avoid the risk of repercussions you should conduct your own security and privacy reviews.
The tools and library review
“The tools and libraries review” evaluates the effectiveness of your system for running “software composition analysis”. These systems tend to report a lot of false positives and unimportant findings. False positives and non-applicable findings need to be actively labeled as such to ensure that the SCA system stays relevant and are used and heeded during the development and testing processes, hence the review. According to Gartner, several vendors of AST systems have started to look into using AI to help their customers to avoid these false positives showing up during analysis. Artificial Intelligence and Application Security Vendors: Marketing Hype or Genuine Hope? You might have to rely on some manual processes if AI not is an option for you, you should, therefore, make sure that the SCA system is integrated into your software development cycle so that developers and testers can address false positives during the development phase instead of having these being reported after the release approval and thereby cause reduced visibility into the your “software composition analysis” findings. To read more about how the tools and library review is setup and conducted, please read my article on “Quality assurance and approval of tools and third party software”.
The security review
“The security review” is needed in order to evaluate “the effectiveness of technical and organisational measures for ensuring the security of the processing”. article 32.1d
Reports from your SAST, DAST, and IAST efforts need to be reviewed. False positives and non-applicable findings need to be actively labeled as such to ensure that the security test systems stay relevant and are used and heeded during the development and testing cycles. Look into using AI to ascertain whether you can avoid false positives showing up during analysis and reporting. The security review should be integrated into the software development cycle to ensure it’s effectiveness.
As with the “The tools and libraries review”, create three documents. One page explaining the purpose behind doing the security review (the why). This is the document you show to your data controller, security advisors, and supervisory authorities. Secondly, create a document describing the step-by-step process including any checklists and routine descriptions (the recipe). This is the document the team will continuously update as their process changes and is used to learn new team members about the process. Then create an overview page where project management can read out the status of each individual review completed or in progress. Then finally, collect and archive all the reviews in a central repository for security and privacy reports according to your ISO 27001 certified ISMS. The security review needs to established guidelines for what should be scanned and when. The tools used for conducting SAST, DAST, and IAST need to be documented together with any manual procedures that need to be done for conducting the review. SAST, DAST, and IAST tools are still considered to be a black art for many developers and testers so new employees need to receive proper training in order to be able to use the security tools properly. All security-related issues found during the review is assigned a work item and addressed immediately or on priority depending on their severity. Deviations found during the review, that also exists in the production environment, which involves personal data and that breaches the confidentiality, integrity, and availability of the system, must be reported to the data controller immediately and to the supervisory authority within 72 hours.
The privacy review
It is a recommended practice to do a manual security and privacy review to “ensure that any weaknesses that could lead to improper use or leakage of personal data are caught. For example, it may be difficult to identify patterns because data alone does not necessarily constitute personal data, but connections between different types of data can provide personal information. In order to ensure data protection, it is important to map where in the software personal data is stored. A review of the code should particularly examine where personal data is written. A common weakness is to write personal data in application logs with insufficient security.” (Norwegian Data Authority, Software development with Data Protection by Design and by Default — Coding)
But relying on a fully manual process for conducting the security and privacy review makes microservice development and continuous delivery impossible. In order to ensure that as much as the process can be automated, Various systems used for information modeling and design needs to be integrated into the software development cycle to ensure that the information models designed by the architect and reviewed and approved by the data controller are implemented by design and by default into the system. There are a couple of ways this can be done. Data input can be validated by Schemas through APIs and at REST by using JSON Schemas or XSDs. Schemas are much easier to acceptance test than code, and they can be exported directly from modeling solutions provided by companies like Sparx to make sure the validations implemented at rest and in motion has been applied according to the design done by the information architect. Data protection by design and by default can then be guaranteed. There is also nothing stopping you from committing the schema directly to a central code repository where it automatically can be picked up by the build process and compiled into the code. MongoDB is NoSQL database vendor that has started with using JSON schemas for validating data before it’s stored at rest in a similar way to how SQL schemas are enforced for traditional RDBMS. If schema validation errors are logged to a central log repository like ELK, ongoing auditing and monitoring of the system can alert on breaches to the DPIA and guarantee the ongoing privacy of the data subject, thereby making sure the system can be continuously delivered and protected by design and by default. Reviewing the privacy of the solution could be as simple as analyzing log messages stored in ElasticSearch.
Smartbear is a company that has a solution for acceptance testing APIs. Their solution can also be used to verify the database result of api calls which means that the tester/developer can verify that data is appropriately encrypted and pseudonymized at REST and that only the data that has been approved for processing by the data controller through the DPIA, is transferred and stored at rest. By combining the capability for exporting schemas from data modeling software with schema validation, logging and API- and database testing, the privacy review can be entirely automated given that some time is set aside for implementing the continuous delivery pipeline from design, development and testing to review and finally, delivery.
Automating your privacy and security review process
To ensure you always are prepared for an audit, you need to be sure you continuously conduct and automate as much of the review processes as possible. If you are developing microservices, you have to. SAST, DAST, and IAST systems used for security analysis and testing must be setup to automatically publish their reports to a central repository where they are made available to personnel conducting the privacy and security review. Make sure you have covered all the points mentioned in my article about Security and privacy testing. The report from the privacy test suite is of particular interest; So is any external penetration test report conducted by an independent offensive security team. The regularity by which you should conduct these internal reviews depends on the degree of automation you are relying upon for doing regular patching and testing of the security and privacy of your system. Only a continuous and automated approach to testing, development, and reporting can ensure that the “Privacy and security reviews” can be automated. I can’t overstate the importance of automation in this regards, especially, if you are developing microservices. Microservice development depends on an “automate or die” approach to ensure continuous delivery. Automating your continuous testing, patching, reporting, and reviews are prerequisites for ensuring that your microservices are properly secured.
AI could help you leverage automation techniques to ensure the correctness of your reporting and the continuous security of your system. According to Gartner, several vendors of AST systems have started to look into using AI to help their customers with conducting application security testing and to sanitize security reports. Artificial Intelligence and Application Security Vendors: Marketing Hype or Genuine Hope?
To conclude
The supervisory authority is tasked with periodically reviewing the operation of the data controller. (article.41) If the supervisory authority decides to audit your operations and find that you’re not reviewing the “effectiveness of technical and organisational measures for ensuring the security of the processing”, they may question your security and privacy practices and fine you appropriately. To avoid the risk of repercussions you should conduct your own security and privacy reviews.
It is a recommended practice to do these reviews manually as “data alone does not necessarily constitute personal data, but connections between different types of data can provide personal information”, but relying on a fully manual process for conducting the security and privacy review makes microservice development and continuous delivery impossible. In order to ensure that as much as the process can be automated, various systems used for information modeling and design needs to be integrated into the software development cycle.
