Reviewing RSA’s 2019 Innovation Sandbox Participants

Hugh Thompson, in the role of MC once again

For the first time in many years, I’m not at the US RSA Conference. It’s expensive and packed, but the vast majority of the cybersecurity industry (especially on the vendor side) make the trek out to San Francisco’s Moscone Center for the show. When I joined 451 Research as an analyst back in 2013, I had never been to an RSA Conference before. It quickly became one of the busiest and most exciting events every year for me.

Analysts often feel pressure to take as many vendor briefings as possible during RSAC, to put faces to names and finally meet many clients, friends and colleagues in person. I always made time to catch Innovation Sandbox though. It is, by far, my favorite part of the RSA Conference. Startups compete, sending in submissions with the hopes of being selected as one of the ten that will compete on stage.

These ten then take the stage in front of a panel of judges (comprised of investors, industry experts and founders) and pitch their company in three minutes or less. The microphone even cuts off when the time runs out! Judges can then ask each speaker questions before jotting down thoughts and scores. After scores are tallied, one Sandbox contestant is proclaimed the winner and receives the title of Most Innovative Startup for that year.

Though I’m watching the show from afar, RSAC gets the recordings uploaded to YouTube within a day or two. I’m going to take advantage of that fact to watch and comment on each Innovation Sandbox performance here, in this post.

If you want to watch through all the presentations before reading my comments, I’ve put together a YouTube playlist. The videos are categorized alphabetically by startup name. I’ve also included videos of the Q&A session with judges after each startup’s presentation. For whatever reason, these Q&A sessions are only available for half of the contestants.

Arkose Labs

Tagline: From suburban Australia to being backed by Paypal: preventing fraud and abuse with 100% SLA

The lone Ozzie in the lineup. Immediate use of Sun Tzu is making me cringe right at the start. Sun Tzu quotes are considered clichéd and distasteful by half of the industry, but it seems like the other half never got the memo. I must admit, however, the quote is very relevant to the topic: “Convince your enemy he that will gain very little by attacking you; this will diminish his enthusiasm.”

“Fighting the never-ending battle of fraud and abuse”

I’m not familiar with the term Single Request Attacks, but he’s saying “9 out of 10 account takeover attacks use this technique”.

They back their tech with a 100% guarantee. I’m assuming they’re intending to do something like a Jeremiah Grossman-style warranty here, but I can’t find any details on their website. They say they have a 2-prong approach: telemetry and enforcement. I’m not clear on how these terms apply in this context.

“Next-gen fraud prevention”

If I were playing a buzzword bingo drinking game with these presentations, I’d already be concerned at this point.

Okay, so it looks like a fancy CAPTCHA and I’m suddenly disappointed. Do we really need a better CAPTCHA? Google’s reCAPTCHA is already pretty effective in my experience, so I’m wondering whether we actually need something else.

They separate authentic and inauthentic traffic… using cookies? How? He’s making the point that computers can’t defeat their CAPTCHAS. They’re the best. 3D. Unbeatable. He says authentic users are never challenged and inauthentic bots are always challenged. How do they accomplish this though? Don’t tell me that and then not provide some sort of explanation or evidence!

Oh, they use machine learning. Of course.

They’re backed by Paypal, which is encouraging, given how much experience Paypal has with fraud prevention and how important it is to them. There’s got to be something to this, right? OR perhaps only the Amazons, Paypals and Alibabas of the world have any need for this ‘next-gen’ fraud prevention and the market is already saturated?

“Make no mistake about it, this is war and Sun Tzu’s proverbs have stood the test of time.”

That was a bit over the top.

The bottom line? I’m unconvinced that they can compete with the multitude of free CAPTCHA options already available. They’re competing against solutions that are good enough for most businesses.


Tagline: “Solving the least sexy part of cybersecurity: asset management.”

I love the honesty!

Hugh just called the company “Axonus”. Close enough?

I’m biased here though, Axonius briefed me a few years ago and I wrote them up. I’m a fan of what they’re working on. I believe that this is still a huge unsolved problem. So big, I feel like, as an industry, we need to pull out all the stops to put this one behind us before we move on. No more machine learning allowed until we solve this one, people!


I don’t know how common this knowledge is, but it’s generally true that most CISOs/security leaders don’t have a good answer to basic asset questions. How many assets are there? What are they running? What’s their purpose? What do they need access to?

He’s saying there are many solutions to handle many types of assets, which actually makes it MORE difficult to answer simple questions, as the data is scattered and incomplete. Knowing our assets is foundational. Sure — it’s the first critical security control for a reason, I think we are all in agreement here and don’t need to be convinced of this. His challenge, in my mind, is to convince us that he can show us something new, or convince us it’s solvable!

Now he’s proving how unsexy asset management is using the Toyota Camry. We don’t need to be convinced asset management isn’t sexy. Has asset management ever been sexy? I don’t think so…

They connect to a ton of other solutions. I’m suddenly having SIEM flashbacks… Wait, they can validate and enforce policies? How? And no agents — this is all through API integrations they’ve built. Maybe some of the integrations are bi-directional and that’s how they enforce?

I like the “show gaps in security coverage” use case, as it should be fairly simple to compare Carbon Black’s asset list with a list of domain-joined PCs from Active Directory. The diff, in theory, should be a list of systems that still need Carbon Black installed. Perhaps GPO isn’t working properly, some corporate-owned systems aren’t domain-joined or the OUs are all messed up.

Time-to-value is supposedly very low, which it should be. This is basically one of those things where you enable all the connectors and should know almost immediately if it is working or not. Time-to-value should only last more than a day if additional parsers need to be built on the fly for unsupported platforms that are required for the POC.


Tagline: “High performance attack protection for production Linux”

I’m biased here as well, as I’ve already been impressed by Capsule8. They’re one of the few startups on this list that I’ve heard of before this Innovation Sandbox competition. The other two are Axonius and DisruptOps.

The vast majority of production infrastructure is Linux… The security industry hasn’t done much with it though

Yeah, well, can you blame us? Windows was pretty bad for a while there. Only now do we have the luxury of worrying about Linux…

SecOps teams are drowning in data; they’re investing in automation, but quality of data source is a problem and monitoring solutions don’t prevent attacks

SecOps is drowning in data? Dude, EVERYONE is drowning in data. The problem statements here and from Axonius are pretty well understood, right? Borderline obvious? I’m starting to feel like one of the women that serve the Mother of Dragons in Game of Thrones: “It is known, Khaleesi.”

Prevent attacks while providing that high-quality data source, runs everywhere, on all versions of Linux

Man, this guy is tearing through this! I feel he’s covered more information by the halfway point than most of the others have in the whole 3 minutes allotted to them. I guess I’m in agreement at this point? I’m not sure what he means about issues with the “quality of data source”. I’m definitely in agreement that any solution here needs to be distro-independent. If you have separate shell script installers for RHES, Ubuntu, SUSE, etc, just shoot me now. I don’t want to deal with that crap. Separate rpm, deb, tar.gz are fine though. Just make it a SMOOTH process. I’ve seen so many commercial Linux products with janky shell scripts.

“Built by black hats to stop black hats”

Entering cringy territory here. Ascending into the cringe-o-sphere. I feel like this is unnecessary these days and we’re past the point where we need to suggest we’ve got ex-cons working for us to impress investors/customers. Later on, he mentions key employees have backgrounds in writing exploits, which I think sounds better than “built by black hats”.

I like that they aren’t just additive — they’re proposing to replace a bunch of older controls, like IPS, FIM and AV.

I like that he got all that out of the way in the first half of his time limit.

They tackle types of attacks, not specific exploits. They created their own “context” to drive down false positives. Works everywhere: cloud, containers, private cloud, datacenters…

“Help companies modernize without compromise”

Overall, very good pitch! Lots of good, useful information! My only additional request would have been a specific example.

CloudKnox Security

Hugh kicks all these off with a company tagline. Some are great, while others are a bit more elusive, like this one.

Tagline: “Cloud Security Platform that automates securing hybrid cloud, starting with managing identity privilege”

Honestly, the talk gets better and by the end, I totally understand the tagline. Lesson learned here? A tagline should make perfect sense with or without the pitch.

“Turned identities into superpowers, which is great for automation…”

Hyperbole aside, he’s right — cloud made it possible to do both amazingly powerful and destructive things with a single command or script with nothing more than an API key.

He’s using VFEmail as an example. It is a very recent example, but not the best example to use for several reasons.

  1. It was a one-man company. Not exactly CloudKnox’s target market.
  2. We don’t understand most of the details about this breach, so he’s making some assumptions that seem unlikely, given what we know. We do know attackers were logged into systems directly, executing destructive commands. There was no hacking at a cloud management/API level that we know of. Besides, it sounds like he wasn’t even using cloud, but rather co-located boxes or VPSs.
  3. Was he even using VMs?
  4. How does he know this attack caused millions of dollars of damage? Where is that number coming from? Who is out millions of dollars because of lost personal emails? I’ve personally done a lot of research into the costs of breaches, so flippantly assigning a dollar value to this event irks me.

Another story of an engineer that shared a proprietary image/container that made it into the public domain. Says that DLP tools didn’t alert on this, because of course they wouldn’t.

“Both of these could have been prevented!”

But I’m failing to see how CloudKnox could have prevented either of these incidents…

“Most identities use less than 99% of their privs on a daily basis”

No argument here — least privilege fixes a lot of problems, but it’s just really difficult and time-consuming to achieve!

“Almost impossible to do this manually”

Ah, so CloudKnox abstracts privs across multiple platforms. Lists AWS, Azure, NSX, vSphere, GCE…

Visibility over who can do what and when; monitor and assess privilege ‘creep’, “take action”: just enough privs controller

“Prevention is paramount and it all starts with identity authorization”

The cloud infrastructure space is jumbled and confusing, so competition will be tough here. You need to have a clear message and value prop. Likely only room for one 3rd party vendor here at the most, as you’re again competing against security controls already built into VM/cloud control stacks.


Tagline: “Elevating your Cloud Security”

I like the auto manufacturing analogy — innovation was building high quality car at scale and he’s arguing that’s what security needs now as well.

“Guardrails for the cloud”

“Impossible to secure all these resources manually”

That’s at least the second time we’ve heard this in Innovation Sandbox — “this can’t be done manually.” Both times so far, it has specifically referenced securing cloud services and components. I also like the idea and metaphor of these ‘guardrails’ that keep you from careening off the edge. I also like that they do active enforcement, not just monitoring and alerting.

Assess, automate, control

Example: Access keys. Can identify unused/stale keys and take action, like notify team on Slack and deactivate the key.

Love that he used an example. It’s cool because it shows how they support the full cycle and can automate it all. Previous vendors in this category (, Paterva, etc) generally didn’t take any action, they just monitored and alerted. It’s also not just security — sounds like some of their guardrails are related to cost optimization/limitation.

“DisruptOps automates security operations for the cloud.”

Overall a very good presentation and they emphasized the right points, like the fact that they address and automate the full cycle for this problem. We have too many products that stop at taking action. Of course, there’s good reason for this — putting any kind of automation that could delete or disable production API keys is kinda scary. You have to make sure you know what you’re doing.


Tagline: “Maximizing data utility, minimizing risk by breaking the performance barrier for homomorphic encryption”

If you’re not already familiar with homomorphic encryption, this one might be tough to follow. It’s a bit of a holy grail, because the idea is that you can use this type of encryption to protect data without making it unusable. Previously, the two were mutually exclusive — if you encrypted something, it became an unusable black box until it was decrypted.

“Gold rush is extracting actionable value out of data”

He’s saying collaboration is necessary for effective ML/AI, but “trust is broken”. We need to protect data while allowing analysis. It’s easy to agree with these statements, but hard to see how effective this approach could be without some specific examples.

He’s saying Company A can run their machine learning model on Company B’s data without exposing either the model or the data, both of which are often treated as sensitive intellectual property.

They won some international homomorphic encryption competition. I like that he actually has analysis time and computational cost included in examples. How do we trust that they can do it without requiring or breaking our own trust though? In fact, a judge asks this! And a second judge!

He has good answers. This contest they won over IBM is one proof point, a lot of their stuff is open source. I think judges were still skeptical by the end, but it’s pretty compelling stuff. They’re definitely one to keep an eye on.


“Defending the foundation of the enterprise”

Not a huge fan of the name — not terribly easy to say or remember. He’s showing us a laptop. Talking about software on it and now talking about firmware implants. They have a firmware implant on a laptop at their booth. Talking about the “hidden attack surface” of firmware. We’ve seen a lot of nasty firmware attacks in the last 10 years…

I get it — “foundation of the enterprise” refers to firmware, because it has code that runs underlying, foundational hardware layers.

Uh oh, choking a bit, forgot his lines, or got really flustered…

Problem statement: firmware attacks are growing. Are they?

So, is this vuln management for firmware? They provide supply chain assurance. Assurance that devices haven’t been tampered with when they first arrive from suppliers and after trips away from the corporate network. So, like FIM for firmware. Doing integrity/tamper checks.

Their product detects and analyzes firmware threats?

Oooh, finishing with some FUD — “how well do you really know your device?”

I’m not left with much after that presentation. I don’t know exactly what they’re doing or how they’re doing it. Is there agent software involved? If so, why couldn’t an attacker tamper with it before tampering with the firmware? I’m reminded of when Bit9 got hacked just so attackers could get past their software using their code-signing keys/certs. I wonder how many companies realizing they’re painting a target on their backs and whether or not they’re prepared to fend off state-sponsored-level attacks.


Tagline: Continuous application security

He looks comfortable. “Most security issues are due to software defects” yeah, okay… “More code = more attack surface” yup, go on… Vulnerability management whack-a-mole (with an actual whack-a-mole animation!).

Code Property Graph (CPG) — okay, he just lost me. I was nodding along, like “yeah, apps have so many bugs!” Then he started talking about representing source code with “hyper-dense graphs” that are “easily navigated”. “Think Google Maps for your source code.” Uhhhhh….

It’s a “semantic graph of your source code”. Doesn’t just include your source code though — it also includes APIs, 3rd party libraries and frameworks. Agreed — I’ve seen stats on how an increasing percentage of code bases are third-party libraries and frameworks these days. In other words, code not written by my developers and probably not vetted very well from a security standpoint.

He’s showing different layers (of the graph?): violations; high-level information flows; methods/types/data flows; instruction level syntax, control flow…

“It can find business logic flaws.”

Wait, what? He’s showing an animation of PII being written to an S3 bucket, representing a potential violation of GDPR policy. How does it find business flaws? He doesn’t explain.

I love examples and he has more of them!

  • Find back doors in router firmware
  • Developer mistakenly exposing credentials via a 3rd party API
  • Find vulns in your own proprietary source code

“CPG is a proven technology — a large fintech is using it to find vulnerabilities 20 times faster.”

Okay, but are they using ShiftLeft to find it 20 times faster, or their own homegrown version of CPG? Also uses examples from a large manufacturing company and a large bank. Sounds like he’s implying these are all early ShiftLeft customers. This really isn’t a crowd that’s going to just nod and accept lines like, “the technology works, trust me!”

Also, if this thing is like “Google Maps for your source code”, where are the screenshots? That sounds awesome! Normally, I wouldn’t expect to see UI/UX in a 3 minute pitch, but when you compare your product to Google Maps, that’s a different story — you just created some BIG expectations!

He’s very well spoken! He’s calm, taking time between statements, clear language, clearly enunciated. Very easy to listen to and understand. Looking around, not pacing like a caged animal.

He’s claiming that, without ShiftLeft, the choice is to fix a vuln or get hacked, which is disingenuous. You can’t suggest that no mitigating opportunities existed before this product. Ever heard of a WAF? Exploit prevention? Yeah, I see where this is going though.

They have a custom microagent (DRINK!) that protects the vulnerabilities that haven’t been fixed. Yeah, we’re already well aware of auto-generating custom vuln mitigations. That’s already a thing. He’s calling it “informed runtime protection”.

He’s saying ShiftLeft changes the fact that AppSec has been a “hodgepodge of tools”, listing SCA, SAST, IAST, DAST, RASP and WAF on a scale with two axes. One is the ability to identify vulnerabilities and the other is the ability to protect from threats.

OMG, the last slide is the ShiftLeft team, but animated like the Star Wars opening crawl and ends with “May the CPG be with you.” My eye just started twitching.

He ends right at the buzzer. He practiced well for this!

First judge question is about in-line patching and false positive/false negative rates. ShiftLeft saying they just do it better, that’s their core IP that they’ve hired PhDs to solve. He says they can do it better because they can read the code.

Next judge: “where’s your extraordinary evidence to support your extraordinary claim of finding backdoors and business logic flaws?” He’s backing off from any suggestion business logic flaws can be found automatically. He’s now saying it’s impossible to automatically find business flaws. The judges sound pretty skeptical, because SAST, in-line patching and other tech displayed here are pretty old by now and haven’t been terribly successful in the past.

I went back and watched the ‘business flaws’ bit and he says, verbatim, “It can find business logic flaws.” Busted.


Tagline: Simplifying and revolutionizing privacy and data protection

It’s funny — I decided to alphabetize these before watching them and it turns out the last startup alphabetically was the first contestant to step onto the stage! Oh well.

He helped lead privacy negotiations for the Obama administration?

He’s saying you can’t just plug in technology to solve privacy because it takes people to solve privacy. We’re solving privacy now? I thought privacy was a state, not an issue. I guess it could be both. He’s saying privacy is an impossible task because the information you need is buried in technical stacks and people’s heads throughout the enterprise.

“We founded WireWheel to offer a different approach.” Simplify privacy to “four critical pillars”: personal data, data store, sharing and processing. And then structure and automate your privacy program to build trust.

“We do it in three ways”

“Wirewheel unlocks the privacy information from your technical stack”

Okay. What?

Plug WireWheel into your IaaS or data stores and they automatically find data, show you where it’s being processed and find personal data. It “translates your technical stack into something your privacy team can actually use.”

Okay, time-out. You can’t just say “plug us into your file servers and cloud and we find all the right data.” DLP has been trying to do that for years and it’s a MESS. It’s common to hear of organizations with hundreds of thousands of DLP false positives. Every day. So how have they solved this? We’ll see if they offer an explanation.

“Second, WireWheel unlocks the information stuck in the minds of people all around your organization.”

Their ‘privacy program in a box’ tells people where to start, what to document and they have a “collaboration engine”. Pricing is affordable enough for small/medium companies and startups.

“Third, WireWheel includes a privacy studio.”

Looks like a customer-facing portal. “We haven’t figured it all out yet”. Talking about partnering. He might as well said, “here at WireWheel, we’re trying to boil the ocean”. Even if you haven’t figured it all out yet, don’t say that — especially after listing off some really broad, ambitious goals.

Ooooh, is he gonna run out of time? Nope.

Not terribly impressed. Lots of huge, implausible stuff they’re biting off at once. Almost no details about how they’re going to accomplish all this stuff. “Unlocking the privacy information from your technical stack” alone is a very ambitious goal and has me doubtful.

Who won?

Axonius won, with Duality as the runner-up. Hugh’s great — he got the name wrong again, saying “Axius” and insisted on a redo for posterity. It was pretty funny. Still, this is a good lesson in branding. It’s more important for a name to be unique and memorable than for it to be clever or descriptive. That said, a challenging name to say or remember didn’t prevent Axonius from winning the day, so take my thoughts with a grain of salt.

Information security veteran blogging primarily about how technology can hinder or help productivity and progress here. Co-founder of Savage Security.