In March 2021, Hootsuite announced having reached Federal Risk and Authorization Management Program (FedRAMP) Authorized Designation. We are extremely proud to have reached this milestone. This article discusses five of the key things that we did to achieve this designation.
What is FedRAMP?
FedRAMP is a US government-wide program for assessing, authorizing, and continuously monitoring cloud products and services. In order to achieve the authorization designation, a service provider must meet a set of criteria, or controls, across a wide range of security related domains. Many US government agencies require FedRAMP certification as a condition of purchase. The FedRAMP designation is internationally recognized as one of the best marks of good security.
There are a lot of resources on the internet that give both high-level and in-depth overviews of FedRAMP. I recommend checking them out to learn the official definitions as well as to get a sense of the program’s scope.
We implemented FedRAMP authorization over a three quarter timespan, with a coordinated effort from Leadership, DevOps, Data, Security, Development, Program Management, QA, Product Management, and Design teams within our organization.
Five Key Steps We Took to Achieve FedRaMp Authorized Designation
As previously stated, the project to achieve FedRAMP authorized designation was a massive undertaking that included every team that had anything to do with software development. Here are the five key reasons outlining why we were able to accomplish this so quickly.
- Buy-in from the top
There are seventeen FedRAMP controls, or criteria, that a company must comply with in order to become FedRAMP designated. Parallel, cross-domain teams were formed to work on the projects. The projects required a high level of coordination and sequencing. Our aggressive timeline left little room for delays.
The executive team was completely behind the project from the outset. The message was clear: FedRAMP is a must-have certification for Hootsuite, and we need to achieve it as quickly as possible. FedRAMP was rated a top priority for all teams. Without top-down buy-in, there would likely have been delays caused by conflicting requirements from other projects. FedRAMP’s tasks were prioritized first, and they progressed quickly.
2. Dedicated experts on our Security Team
Obtaining FedRAMP certification is a difficult task. The FedRAMP controls must be understood and interpreted in the context of our own SaaS service. Certification is subject to independent external review, and there is no wiggle room allowed.
Our development team received security training to get a good foundation, but that was just the beginning. It was the professionalism and experience of our security experts in the organization that made the difference. Many of them had worked on delivering FedRAMP in other companies, and they had a deep knowledge of the FedRAMP controls. They assisted the development team throughout the project by guiding us and pushing us to a better understanding of the controls and how to implement them. The project’s north star was our security experts. They guided us in the right direction and ensured we understood how to get there.
3. Consolidation and Simplification
We are a reasonably mature SaaS company with processes for developing and delivering software. In order to prove to an external auditor that we adhered to FedRAMP security controls we needed a higher level of formality, visibility, and predictability. We had the opportunity to review our software and deployment processes and consolidate and simplify them so they would be easier to manage.
For instance, were there any partially manual processes that we could fully automate, was there a slow pipeline that we could speed up, or a non-standard pipeline that we could standardize, were any of our metrics outdated or unclear, were there any code or services that we no longer required?
An example of this in action was how we enhanced our Software Composition Analysis (SCA) scanning process. Our architecture is based on microservices, and teams are typically responsible for maintaining their own software. It led to a lot of duplicate effort when upgrading common third-party libraries. We changed the process so that one team provided a common bundle of libraries that were SCA compliant, and tooling was added to automate the pull requests for the bundle across all services. Besides reducing duplicate effort, this also had other benefits, such as standardizing library usage across the teams and reducing cognitive load on developers.
Similarly, we standardized UI deployment pipelines, improving standards, and reducing maintenance costs.
Whenever we upgraded software, we asked ourselves two questions: Do we still need this in our product? And if we do, does it need to be implemented this way? As a result, a long list of improvements were made, here are the highlights.
- Many of our databases were moved to a managed model to reduce management overhead.
- Deployment pipeline orchestration was moved to Jenkins on Kubernetes for the reason above.
- Deprecated features and services were removed.
- Thousands of lines of dead code were removed.
- Tens of thousands of static compile errors were removed.
Although we knew we wanted to upgrade our processes, we did not know exactly what would work best for us. We adopted an experimental phase in those cases so that we could test different solutions. We would brainstorm ideas at the beginning of the project and rank them. Next, we ran experiments on the different ideas, and then ranked the results of the experiments using a simple scoring method based on RICE scoring. We came up with an optimal solution, which we then implemented.
When we were looking to automate microservice software upgrades we tried six different approaches before the optimal solution was selected for our teams and implemented. Several experimental techniques, such as A/B testing, are still being used in non-FedRAMP related work.
Our development teams work independently for reasons of efficiency and productivity, and are responsible for their roadmaps, software stacks, etc. Several FedRAMP projects cut horizontally across our development team. This meant that in order to fully comply with some FedRAMP controls, our entire development team needed to adhere to it. Compared to a typical feature development project, we needed to do a lot more cross-team collaboration.
We achieved this by increasing our collaboration over shared communication channels, sharing delivery goals, and tracking status by attending joint planning meetings. In order to address common engineering problems for multiple teams, we set up temporary tiger teams composed of members of multiple teams. It was a great example of the Hootsuite #oneteam philosophy in action.
FedRAMP has been a net benefit for our software development team. Here’s what it did:
- Improved almost every aspect of our technology platform.
- Reduced technical debt.
- Increased efficiency in deploying, maintaining, and developing our software.
- Contributed to continually raising the bar for software quality.
- Provided us with a new technical foundation on which to build in the future for more extensive architectural changes.
- Created the opportunity for our development team to experiment, consolidate,simplify, and collaborate more effectively as a result.
- Showcased the value of obtaining executive buy-in for projects and the value of hiring experts in specialized fields.
Additionally, one short-term downside to the project for the sake of balance. Is that we probably spent more time on infrastructure and tech department projects than we had originally planned, which temporarily reduced time for some greenfield development.
As our customers appreciate, we are already back to concentrating fully on improvements that are more visible. Based on our experience, the short-term focused efforts have already paid off and were well worth it.
For our development team, the FedRAMP project was an amazing success. We are thrilled to have reached this milestone, and we are already working on many new projects that are built on top of our improved codebase. This information may prove useful to you if you are embarking on your FedRAMP journey. I wish you best of luck.