The ratio of engineers in Development, Operations, and Infosec in a typical technology organisation is 100:10:1

The above statement holds true most of the time. When security professionals are outnumbered, without automating or integrating security into the daily work of Devs and Ops, Security teams might only be restricted to do Compliance testing and one time pen testing which incurs huge costs but might not serve the purpose of delivering a secure application.

One of the most quoted terms in the world of Information Technology to tackle such a problem is DevSecOps which is derived from DevOps by including security to it. While that is how things should be DevOps should inherit security to it. But, how do we go there?

The idea is to shift security towards the left of the SDLC. This will help teams to come together and talk about security more often. This will help in “delivering secured features in less time to market” as security comes packaged with every phase in SDLC. Start with simple things like

Integrate Security into Sprint/Iteration

Making small changes to incorporate security related activities into our daily activities helps DevOps consume Security

• Performing Iterative Threat Modelling on the application you are working on

A Guide to Threat Modelling for Developers By Jim Gumbley is a good start for Threat Modelling Aspirants

• Involving security professionals(People with security knowledge and background) from the initial phases of the project
• Talking about security related stuff in Sprint demonstrations/ Iteration showcases
• Adding security related acceptance criteria to all the business stories/ technical stories
• Add evil/abuse stories

Integrate Security into Project Tracking or Defect Tracking Tools

We generally tend to see all the security related issues in a large PDF report after a pen testing or stored in a GRC(governance, risk and compliance) tool and often tend to ignore putting them on to our project wall like JIRA.

• Converting the outcomes of threat modelling in to stories and add them to the project wall
• Tracking all the security related stories/defects/tasks on project wall
• Writing security specific stories if necessary(recommended only if the scope cannot be incorporated in any of the existing stories, coz security stories not get prioritised )

Integrate Preventive Security Controls in to Shared Source Code Repositories and Shared Services

We create so many accounts in a project like GitHub, Jenkins/CircleCI, Splunk etc. This will make our life easy in the world of DevOps to be collaborative and at the same time this deals with a lot of sensitive information like source code, Cloud account keys, logs and so on. When we try making our life easy, it is important for us to make it secure as well. The security teams can help them by providing a list of security services or libraries that their environment requires, such as user authentication, authorisation, password management, data encryption so on and so forth. So that teams can choose to use them as per their needs.

• Using a centralised shared services organisation and implement SSO for all the applications with MFA (for Git, continuous integration environments like circleCI or gitlabci)
• Providing a list security services/libraries/tools to all the teams and configurations (logging, 2fact auth library, etc)
• For Secret management (connection settings, encryption keys, cloud account keys) using tools like vault, creadstash, Keywhiz, AWS secret manager
• Using Team password manager options like LastPass or 1Password.
• Providing training for teams using these services and libraries for the first time.

Collaborating with the Ops specialist on the team to make this process easier to everyone to make this process easier to everyone on the team, automating things wherever possible also helps in people picking up these things without a lot of setup required for everyone to go through.

Integrate Security into Our CI/CD Pipelines

The era where security reviews and pen testing is done at the end of the development and getting hundreds of pages of vulnerabilities in a PDF which will be handed over to development and operations, which in turn will mostly be ignored due to delivery milestones. One way to prevent this from not happening in the time of DevOps is to automate security on the pipelines and these would run on every commit.

Beware of what you push

While developing we deal with a lot of sensitive data like passwords, credentials, AWS cloud keys, etc and these are essential for us to make the code work. Sometimes unintentionally we might push these secrets into our source code repositories. We can prevent such things from happening and can use tools like Talisman, Hawkeye file-secrets, gitrob to our rescue.

Talisman is a tool to validate code changes that are to be pushed out of a local Git repository on a developer’s workstation. By hooking into the pre-commit & pre-push hook provided by Git, it validates the outgoing change set for things that look suspicious — such as potential SSH keys, authorisation tokens, private keys etc. The aim is for Talisman to scan both file names & file content so that even potential problems embedded in source code and documentation can be caught.

Static Application Security Testing

Static Code Analysis (also known as Source Code Analysis) is usually performed as part of a Code Review (also known as white-box testing) and is carried out at the Implementation phase of a Security Development Lifecycle (SDL). Static Code Analysis commonly refers to the running of Static Code Analysis tools that attempt to highlight possible vulnerabilities within ‘static’ (non-running) source code by using techniques such as Taint Analysis and Data Flow Analysis.
Tools: Checkmarx, tool list

Scanning Vulnerabilities in Dependencies

Developing a software involves a lot of things, one of them being heavily used is frameworks, libraries etc., which are dependencies. These dependencies are nothing but a piece of code and they too have a chance of being vulnerable. Dependency Scanning helps to automatically find security vulnerabilities in your dependencies while you are developing and testing your applications.
for example: when your application is using an external (open source) library which is known to be vulnerable.
The idea here is to have them running on pipelines.
Tools: OWASP Dependency checker , synk

Scanning images for vulnerabilities

In the era of docker and kubernetes, use of containers is on a rise. Most of the applications developed these days are based on container technology. If you are using this, then definitely you will have to use a container repository to push your images. At this level various tools offer to scan these images on push this can help you automate detection and responses to container image vulnerabilities prior to promoting and deploying into production.
Tools: Docker’s inbuilt ability to scan, AWS ECR image scanning, so on..

Dynamic Application Security Testing

As opposed to static testing, dynamic application testing consists of tests executed while a program is in operation. Dynamic tests monitor items such as system memory, functional behaviour, response time and overall performance of the system. This might be in a similar manner on how a third party user will interact with the system. There are tools which can be used to find security vulnerabilities on a running software/application. This can be automated on the pipeline.
Tools: ZAP, Nessus ., etc

Source code Integrity and code signing

Everyone who has access to code and permissions to checkin so should have their own PGP key, perhaps created and managed by a system like keybase.io . All commits should be signed. This is straight forward with tools like git and gpg.
Every artefact that is produced during by CI pipelines should also be signed and the hash should be recorded in the central logging service for auditing purposes.

Check this article by Bilal Fazlani

Password vaults or secret managers

In the devops world, we have all the code and systems running on the cloud. There are a lot of secrets(passwords, tokens, cloud environment keys) that are being floated to make these systems work. These should always be managed by password vaults and not be stored in any config. One of the advantages of maintaining vaults is the rotation of the secrets can be automated.

Tools like Gitlabci will give most of these services inbuilt if you are using such tools .
look out for what they offer in this space.
Circleci too have some things inbuilt to support devsecops.

While we talk about things to adopt when we try to include security in devops, there are some things that we need to keep in mind while transitioning to such a development model

• Prevent security mistake from being repeated
• Integrate security objectives into the development tools
• Maintain fast flow through infosec, through automation
• Ensure security of the environment(Harden the environments, enable necessary security settings on cloud)
• Integrate security into production telemetry, create security telemetry like unauthorised login attempts, password resets, credit card changes
• Monitor the security telemetry to detect any kind of attacks being performed
• Protect your deployment pipelines, they are prone to attacks too
• Providing training for teams on the practices in security in SDLC. Like how to perform threat modelling to the development teams

If anyone can answer what is the most secured house on earth?
Secured house is one which doesn’t have any doors or windows or an entry point. Which is impossible! Or such a house is good for nothing. Rather we build a secure house. So let’s build our tools & products securely. Since we have more attack surfaces now, including these security measures in devops is very essential to build a secure application.

References:

[1]: The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organisations.

[2]: https://about.gitlab.com/solutions/dev-sec-ops/

[3]: https://owasp.org/

[4]: https://martinfowler.com/articles/agile-threat-modelling.html

[5]: https://medium.com/weekly-webtips/protect-yourself-from-git-identity-theft-f6836f3249d5

Recently, I had to identify a error/validation message on form submit in a application with selenium webdriver. Like every other element, I tried inspecting it the error that’s being popped out very similar to a tooltip. I couldn’t find anything in HTML or DOM to identify the error/validation message. Wondered how this is happening, later figured out the HTML5 provides “required” attribute, which lets the browser to throw a error/validation message.

What is this HTML5 “required”?

The Boolean required attribute marks any form control as being required to have a value before the form can be submitted. In browsers supporting constraint validation, any fields with this attribute which lack a value will prevent the form from being submitted.

<input type=”text” name=”username” required>

How to identify the element?

By implementation the browser is the owner which will throw the error message. Here with HTML5 “required”, the browser is not injecting any code or changing the DOM. Error message can’t be identified because of this.

The only way we can identify the error message is possible if the developer has used constraint validation API to provide a custom error/validation message. This way the DOM will give the error/validation element on page.

How do I verify the error/validation message?

1. If developers are throwing a custom error/validation message the element will be in DOM, using any of the locators(CSS, Xpath, id, name.. etc )
2. If developers are using default error/validation message thrown by browser.
   a. We can use the mouse actions to moveTo the element and do getText() and verify.
    — This is flaky because any other action might let the error/validation message disappear.
   b. We can verify whether the attribute “required” is present in the HTML of the element.
    — We are dependent on the HTML5 behaviour of the browser. So we need to verify the support of the browser with the client side technologies whenever there is an update on either side i.e., browser or the technologies we use.

isAttribtuePresent(webelement, required)

private boolean isAttribtuePresent(WebElement element, String attribute) {
Boolean result = false;
try {
String value = element.getAttribute(attribute);
if (value != null){
result = true;
}
} catch (Exception e) {}
return result;
}

Learnings:

Understand the tech stack and how the changes are gonna effect automation that we are doing on the application.

Keeping an eye on what versions of the browser the application is compatible with is very much necessary. We might tend to miss this on legacy applications.

Verifying code in automation and leveraging the efficiency to do while trusting the third party technologies or libraries that we use.

More resources:

https://www.html5rocks.com/en/tutorials/forms/constraintvalidation/

https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/HTML5/Constraint_validation

https://www.wufoo.com/html5/attributes/09-required.html

It’s not doing Agile. It’s being Agile

Talking to a friend about a trip.
Friend1: Can we postpone the trip by a week.
Me: Let’s see how it works with others.
Friend 2: Sorry, I can’t take leave next week. My team needs me!! we have lot of work to finish off for this release.

Digging deep into the details, why such a situation in the team. I came to understood that the estimates on the project are not realistic, the entire team thinks the same that - they can pull it off.

I was surprised to understand, knowing the fact that the estimates are gonna put the team in trouble how everyone has agreed to go ahead and test the waters!!

I have often seen this in agile projects, how wrong estimates have got the hands burnt at the end of deadlines for releases.

Which leads to:
Working more than required — Team has to stretch
Convey the client and extend deadlines — Creates loss of trust on the team
Work on weekends — Team starts disliking the project
Manage to achieve velocity now — Client expects same going forth
Many more..

IMO, it is not a situation that a team should be in!!

Being in such a situation decreases the productivity of the team, rather than increasing it.

All the above said methods are like patching the project for that specific point of time.

Try to identify the root cause and fix it:

Din’t get correct estimates?
One of the most common problems that teams face: not getting the right estimates. This happens quite often, but understanding this problem and getting that fixed plays a crucial role.

Speak out if you feel that estimates are gonna put the team in trouble
Understand yesterday’s weather and keep correcting estimates time to time
Promise what can be delivered and meet it, rather over committing and failing

Velocity changes?
Expect the unexpected.
Consider anything that might impact on velocity while planing a release. A team members marriage or third party dependencies (co vendor or an open source product) get a contract on turn around time.

New team member ramp up time
Team leaves across the release
Risks and third party dependencies
Keep client updated with all these impacts

Changes in iteration once IPM is done?
Best way to deal with things like this is to put a practice in place. No changes to stories once stories agreed for the iteration. If it is a should from client then drop something from agreed stories. If we don’t do this, then it is gonna result in a scope creep.

Set a practice with client that the team is gonna follow
Don’t change iteration wall(stories) once the iteration is started
Again convey the impact of the changes made to client(if the changes are unavoidable)

Blocked on something?
If ever you have come across a blocker, communicate to client then and there. If needs a discussion from client IT team, there is a dependency on another component from other vendor.. so on..

Always do the right communication at the right time
Keep tracking RAIDs on a common medium

Out of everything, set practices and communicate to client and follow them. We are humans and we do forget. Keep reminding ourselves about what we have set up. We are humans and we do mistakes. How quick we learn from them is all that makes a difference.

Let’s not patch! Let’s Fix!!

P.S.: Everything above is in my opinion, any feedback is welcome :) Happy to learn!!