What we learnt scanning 10.2 billion lines of Salesforce code

We find one serious security problem every 39,000 lines of code; this affects, on average, 2 orgs out of 5.

Lorenzo Frattini
Jun 12 · 4 min read

At Clayton, we examine tens of millions of lines of code and configuration every day, looking for security vulnerabilities and anti-patterns before they put at risk our customer’s Salesforce investments.

We have recently surpassed our first 10 billion lines of code, which is a fantastic milestone for a small startup like ours. We thought it was a perfect time to look at some of the things we have learnt about the Salesforce ecosystem, and share some insights from our unique perspective that might be useful to many of the experts out there.

How common are security problems?

Let’s start with the bad news. On average, we find one serious security problem every 39,000 lines of code; this affects, on average, 2 organisations out of 5.

The Salesforce platform has a wide array of very solid safeguards to protect customers data and is generally exceptionally robust in many respects. However, when one starts customising their Salesforce org with code, security problems can naturally emerge. And indeed they do.

Data leaks: the most common security risk

Our data shows that the most common issue is by far the risk of data leaks, which affects about 30% of the orgs we assess. This incorporates a number of problems such as CRUD, FLS bypass and Insecure Direct Object References. In the last 6 months alone, we detected this problem almost in 3,000 different circumstances: CRUD/FLS bypass was the culprit in 72% of the scenarios we analysed.

These problems are tricky to detect: they typically arise across different parts of the source code and need different parts of the code to be assessed at once (i.e. views, controller and data access layers). These problems are hard to spot for humans and largely invisible to code linters, which is what makes them go unnoticed so often.

The good news is that Salesforce is doing a lot in this direction: in addition to decent training material — such as the Data Leak Prevention module on Trailhead — technical improvements like inherited sharing and the possibility to easily strip away inaccessible fields go a long way to helping developers avoid these problems.

The bad news is: developers still need to pay attention to it and make sure they address it properly, to avoid putting customer data at the risk of being inadvertently exposed. Which obviously would be a bad thing.

ISVs are much better at addressing security problems

If you are Salesforce ISVs, your apps are vetted regularly by Salesforce to ensure they stay compliant with Security Requirements — this is extremely important to ensure the overall security of the AppExchange, and Salesforce is very serious about it.

Salesforce customers, on the other hand, have no such requirements: the overall security of their customisations is “at their own risk”. While this may seem reasonable, data suggests that end users aren’t so scrupulous about addressing security problems. According to our data, end users are 85% more likely to prioritise new features over fixing any insecure code.

Naturally, when investing in Salesforce, delivering value for the business is a priority, so one could see why product owners might prioritise new features over security. However, with an ever-increasing amount of customer data being stored in the cloud, security is today more important than ever, and getting it right should be a priority for every organisation that is truly committed to being customer-centric.

42% of implementations use dummy unit tests

Salesforce enforces certain test data coverage to every organisation, which is a very good thing to increase awareness about robustness and good testing practices in the ecosystem. At least, this is the most natural conclusion to draw.

Our data, however, suggest that reality may be a bit different. Although test coverage is generally high (having at least 75% test coverage is a requirement in order to deploy any code customisation in Salesforce), having unit tests that don’t actually verify the expected behaviour is surprisingly common.

42% of the implementations we assess show a pattern of using tests just to achieve coverage, instead of actually validating the logic. This malpractice might be a legacy of an old way of doing things (I always see a great number of talks about good testing practices at every Dreamforce, World Tour, meetup and community event I attend). But many companies, still to this date, are being slowed down by the burden of poor testing practices.

If your Salesforce development is slowing down instead of speeding up, this may be the reason.

Salesforce developers are 57% more likely to address performance than robustness.

Our data suggest that performance problems are often prioritised over robustness: they are resolved more regularly and tackled more quickly, time to resolve is 2 hours less on average.

Obviously, performance is very important when building on the Salesforce platform, especially considering the impact that governor limits have on the overall user experience, adoption as well as scalability to large volumes. On the other hand, however, robustness has a long-term payoff: it makes defects less likely as complexity grows, and contributes to increasing the overall agility and make companies faster in their execution.

In conclusion

These are just some of the things we have learned by analysing Salesforce code and metadata at scale. Are you a Salesforce expert? Do these insights match your personal experience? Leave your comment below.

Try Clayton for free

Clayton is the most comprehensive, accurate and simple Salesforce code assistant, loved by companies like Deliveroo and Sapient i7. If you are interested in knowing more about Clayton or would like us to assess your Salesforce implementation, check out https://try.clayton.io

Lorenzo Frattini

Written by

Lorenzo is a Salesforce Certified Technical Architect and the founder of Clayton. You can find him on Twitter at @lofrattini.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade