Enhancing vulnerability prevention through human-based security measures

By Nicolas Béguier, Security Architect

As one of the first security guys at leboncoin, I was involved in building the first proper security team when we hit 1,000 employees a few years ago. Today, this security team has 5 people working for it, with me as the security architect. We protect infrastructure, IT devices, and products (along with other security teams at Adevinta, the company that owns leboncoin).

Our first security measures to protect our products were anti-bots and anti-DDoS solutions. But they weren’t enough as they were only protecting production. So we decided to add code scanning, vulnerability scanning, and pre-release pentests, among other things, to find out if the code was secure before it was released. But that still wasn’t enough: We got involved too late in the development process, so we were only able to pentest 1 new feature out of 10. And with 600 people on the tech team and 60 feature teams, limiting vulnerabilities in source code was getting harder.

That’s what made us want to enhance human-based security that would prevent vulnerabilities while the features are still only in the product team’s heads and not coded yet. So the purpose of this article is to share the human-based security measures we’ve put in place at leboncoin and what we’ve learnt from them.

Integrating security into the software development life cycle

In order to detect vulnerabilities efficiently, security measures are needed throughout the software development life cycle (SDLC), not just when a feature starts to be built.

Production

At leboncoin, the technical security measures to protect the source code already in production consist of:

• Anti-bot solutions
• Anti-DDoS solutions
• A rate limiter
• Strong authentication

As well as costing a lot of money, security measures put in place to fix a vulnerability in production also have a negative impact on roadmaps. Pushing security fixes after a release also requires more energy.

Build, Test, and Pre-release

At leboncoin, the technical security measures to detect vulnerabilities before the release of a feature comprise:

• Code scanning
• Vulnerability scanning
• Pre-release pentests
• Dependency scanning
• SIEM
• Bug bounty

These security measures are important, but they are still limited. A code scanning solution may not find any vulnerabilities in a feature because it only focuses on bad code writing (which is rare in a mature company with peer reviews).

Discovery

At leboncoin, the human-based security measures used during the discovery phase consist of:

• Product review
• Threat modeling

Discovery doesn’t have any code yet; only mock-ups are being developed. However, security measures can still be applied at this stage.

Pre-discovery

At leboncoin, the human-based security measures used during the pre-discovery phase comprise:

• Awareness
• Guidelines

The purpose of awareness is not to teach developers and the product team security, but to teach them where to find guidelines or how to ask for them.

In the following sections, we will discuss leboncoin’s human-based security measures in more detail.

Human-based security measures: what we’ve implemented and what we’ve learnt

Maturity, awareness, and guidelines

Security maturity involves determining what the current security measures cover and what they don’t.

When maturity is established, guidelines can be written to raise awareness among employees so they know what the security issues are. But be careful: You cannot force people to do security. No matter how hard you try, it won’t work because it’s not their job, and that’s fine. You should not expect them to know what an XSS is, but for them to be able to answer the question “What could go wrong?” Leboncoin offers a global awareness program for all its employees today. We plan to add a product team training program soon, but we are already running an additional specific awareness program for developers.

A few years ago, I organized some workshops to explain what XSS and SQL injections are to developers. These events taught me that developers should be able to use their day-to-day languages and frameworks to make this work. That’s why today we hold “Security Dojos” during which we focus on a specific topic and practice offensive and defensive measures. During these workshops, developers are asked to add an SQL injection and then fix it using their usual framework. It’s sometimes difficult for them to do that because they don’t understand what an SQL injection is. In some cases, it’s difficult to bend the framework because it does not want SQL injections. That’s OK, though, because that’s the whole point of the experience.

In order for guidelines to be efficient, you must know what you expect people to learn in order to determine what to focus on. Be technical for technical people and stay close to developers’ reality with code examples and libraries. Your explanations should be as succinct as possible — putting everything on page is easy, but if it’s too dense, people won’t read it.

Product review and threat modeling

The dedicated Slack channel

During the early days of the security team, we set up a Slack channel for people to use to warn us two weeks before new features went live. We wanted to give ourselves some time to do pentests before the features were in production. For two years, we tried this method, but it didn’t work, mostly because it wasn’t enforced properly: There was no carrot or stick. In addition, our intervention came too late in the feature development process — everything had already been decided by this point. The channel was kept open, but we developed other ways to know about upcoming features in parallel, including holding meetings and drawing up a checklist.

Meetings with feature teams

One year ago, we began conducting product reviews by holding one 30-minute meeting every quarter for each of the 60 feature teams (including engineering managers). In a way, it worked well because we caught every ongoing feature before it was developed. The downside was that the minutes writing and meetings themselves took up a lot of time. So instead we decided to have the product team complete a checklist asynchronously. Currently, we still hold meetings with feature teams who do not complete the checklist, which represents approximately 50% of the teams.

The checklist

The checklist was implemented at leboncoin two quarters ago and consists of a short form that the product owners and engineering managers must fill out at the end of the discovery phase. Its purpose is to both obtain information about features that will be developed and raise awareness. A wave emoji 👋 attached to some of the answers indicates that the security team needs to ask more questions. Product people will know even before filling in the form what situations may cause security concerns.

Questions

Checklist questions include:

• Does your feature involve a third party?

In most cases, people want to use free versions of third-party software. A legal contract is always required in these cases.

• Are you using a known authentication method or is it a new one?
• Is there any user input?

This question is difficult since everything could be considered user input: When someone interacts with the application by providing data, it is considered user input. And it is user input that breaks most applications.

• Are you going to manipulate or ask for any public, confidential, secret, or business data from the user?

A few additional questions are asked if it is not public data.

Product owners and engineering managers will know if the checklist doesn’t contain any concerns before sending it back. In the event of a warning, they should wait for us to get back to them. After that, we perform threat modeling and, if necessary, a pentest.

Checklist 2.0

We’ve made some adjustments along the way to make the checklist more efficient (and we’ll keep improving it). After testing it for two quarters, here are the lessons learnt:

• Make the checklist as short and simple as possible

Your checklist will not be filled out properly if it is too detailed. In addition, keep in mind that you will always need to ask more questions since a checklist can never cover everything.

• Communicate in the language of the product team

Avoid asking questions that are too technical and check with the product team to see if they understand the questions. The formulation is key here, and remember that it is not the questions that matter, but the answers.

• Bring awareness

Wave emojis are a good way to raise awareness about potential vulnerabilities.

• Context is important

In addition to getting the name of the feature, along with a brief explanation, it is also important to know the name of the team or the Slack channel that will be used for discussions. Especially in a large company like leboncoin.

Threat modeling

There are many ways to do threat modeling. In essence, it involves mapping each risk associated with a feature. The exercise is challenging because you must consider everything that might happen. It is necessary to understand the feature well, which requires close collaboration between the security team and the product team. For example, it might be about finding out what could go wrong with a payment — what is the best way for an attacker to withdraw money from wallets and transfer it to their own bank account?

In some cases, we have also learnt from attacks that our competitors have received. Several weeks ago, people broke into Vinted accounts and emptied every wallet containing 100 euros or more. To determine whether the same attack would be possible at leboncoin, we raised an incident. Following that, we worked with the product team to model threats. For each step in the attack, we evaluated the risk of being attacked and whether we were willing to take it. There is a tricky balance between making sure that end users’ purchase experiences are fast and safe at the same time. In the end, it’s all about trade-offs.

So far, so good

Although our current process isn’t perfect, our new combo of human-based security measures — meetings and the checklist — has allowed us to be aware of 90% of the new features.

Our preventative security measures work today because we fit into the product team’s language and rituals. We have had to adapt to put security within the product team’s processes, but let’s be honest, it wouldn’t have been a success without management pushing security measures that will obviously slow down new features.

If I had to do it again, I would focus more on awareness for the product team. My belief is that if we had started on this a few years ago, we would have better results today. Most security teams only care about providing security knowledge and don’t consider employee ability in this area. And that’s why product awareness is so complicated for them.

So that’s it! Hopefully this article will help you come up with the best security measures. I’ve enjoyed sharing my insights into human-based security measures and laying one of the first stones of this topic.

Written by Nicolas Béguier, Security Architect