Security & Infrastructure at Fortmatic

Sean Li
Fortmatic
Published in
13 min readNov 15, 2019

Background

We started building Fortmatic in June 2018, and beta launched our minimal viable product in February 2019 during the ETHDenver hackathon. We couldn’t afford to build the wrong product, and want to get started iterating with developers quickly to stay on top of such a rapidly changing ecosystem. We had to make many pragmatic decisions, balancing between challenging priorities, especially around our key management architecture. We had to leverage our team’s past expertise securing internal sensitive data, secrets, and infrastructure, and therefore, the initial version of Fortmatic key management adopted a traditional Web 2 custodial model where users’ private keys are generated and encrypted within a secure enclave in our infrastructure.

Since our launch, we’ve been surfing a tornado… iterating with developers, partners, fighting fires, and prioritizing features for our roadmap. The incredible amount of traction from developers looking for alternative key management approach caught us by surprise, and we’ve learned so much about the essence of Web 3. From working closely with developers and partners, the most significant criticism we faced was the traditional custodial key management model. Since launch, we’ve been working with over thousands of developers and users on pushing the boundaries of key management, exploring ideas on what’s acceptable and what’s not. Finally, after months of building and testing, we launched our 1.0 version and rolled out our new non-custodial Delegated Key Management architecture to mainnet on 11/4/2019.

Our philosophy on key management

Here at Fortmatic, we adhere to a humanistic and pragmatic philosophy on key management. Transformative technology always resonates with user behaviors and aligns involved parties’ incentives. To make steady advances in key management, we have to assume that human behavior is indeterministic, both users and attackers will eventually evolve and exhibit different behaviors. We summarize our philosophy into three pillars.

Accounts need to be recoverable. With a large enough audience, users will lose their device, recovery kits, and forget their passwords or recovery questions, etc. One of the worst things that can happen to users is them getting locked out of their accounts. Users will also be able to export or import their keys at any time — giving them complete control and ownership of keys.

Security needs to be extensible. We can’t lock ourselves into an inflexible or unscalable architecture to the point that we cannot support a large number of users and quickly react to changing user/attacker behaviors to patch or upgrade our system instantly. Fortmatic keys can also be used in conjunction with smart contract wallets to introduce more layers of security.

Company needs to be accountable. Companies provide value by taking responsibility and serving customer needs. There are many potential ways for companies to be evil (consumers are skeptical), which means trust is always required between company and customer. Building trust is a long-term process and investment, and Fortmatic aims to be as transparent, auditable, and have mature security controls (SOC 2 compliant, penetration tests) as possible.

The design behind the new Delegated Key Management architecture

Our new key management architecture is called Delegated Key Management (pat. pend. USPTO 62/904689), as it delegates encryption and decryption operations to a trusted 3rd-party service. In Fortmatic’s case, we’ve entrusted Amazon Web Services’ (AWS) Key Management Service (KMS) and Cognito. At a high level, what this helps us accomplish is that users can directly interact with AWS’ KMS on the client-side — completely bypassing the Fortmatic backend for encryption and decryption operations, non-custodial since we’ll never “see” users’ private keys, and therefore improving our trust among developers and users.

Leveraging hardware security modules

To achieve this, Fortmatic leverages Hardware Security Module (HSM) provided by AWS’s KMS. On the HSM, it stores the dedicated user master keys, which are generated using AES-256 with 384-bits of entropy. The master keys never leave the hardware as they are meant to be created and stored in there. No one can export them out either. The entire encryption and decryption operations happen inside the hardware itself. Think of the HSMs like hardware wallets such as Trezor or Ledger, but in the cloud, secured by AWS’s data centers.

How client interact with AWS

For users to access their HSMs, they authenticate with the Fortmatic relayer. Upon successful authentication, users receive a time-bound access token, which is traded for scoped credentials, and in turn, allows the client to interact with AWS directly. The scoped credentials exclusively enable users to call the AWS to access their master keys stored on the HSM for encryption and decryption. This mechanism bypasses the Fortmatic backend entirely, and Fortmatic cannot forge nor intercept the scoped credentials because they are generated by the operations exclusively between the clients and AWS. Both access tokens and scoped credentials are created dynamically by AWS, with audit logs, and with TTLs enabled.

When a user signs up, a public-and-private key-pair is generated on the client-side for the user inside an iframe, inaccessible by the integrated app. Keys are generated with cryptographically secure pseudo-random 256-bit of entropy. Once generated, the key is then encrypted by the delegated key management solution, where Fortmatic never sees the raw private keys at any part of the flow. After encryption of the keys with the HSM, the encrypted key is then stored on the client-side iframe as well as uploaded to the Fortmatic relayer for redundancy, in case the encrypted key on the client-side is wiped out. When users authenticate again, the encrypted private key will be downloaded to the client. Users then can decrypt the encrypted private key directly with AWS, and all Web 3 primitives are done on the client-side bypassing Fortmatic backend. Fortmatic does not know the content of the raw private key, nor does it know how to construct scoped credentials to access the users’ dedicated master keys stored in the HSM.

New user key generation

User signing transactions

Account recovery

Powering smart contract wallets

Instead of offering full-fledged wallet experience to users, Fortmatic aims to provide an accessible identity and key management service to developers. The Fortmatic UI widget is a great way for developers to get started and serve their users right away quickly. We believe in letting decentralized applications (dApps) completely own their user experience end-to-end, instead of depending heavily on 3rd-party wallets’ UI, which may dilute dApps’ brand and result in drop-offs from mistrust and jarring context switches. With Fortmatic staying in the background as an infrastructure layer, dApps can maximize their conversion rate, cater to their own audience’s needs, and build a product that rivals the ease of use of regular Web 2 applications.

Hence, we’ve opened up registration to Fortmatic Enterprise. We are offering white-labeling (make our UI look and feel like your dApp), headless mode (key management service without our UI), as well as custom integration with popular smart contract wallets like Gnosis. Developers will be able to use Fortmatic’s SOC 2 security compliant identity and key management service to provide one of the keys for smart contract wallets. This will enable functionalities like multi-sig, social recovery, dead man’s switch, withdrawal limits, and meta-transactions — significantly raising the level of trust, security, and usability. DApps will be able to bring their own smart contracts that cater to their own user-base.

Migration of older users

With our new Delegated Key Management architecture, Fortmatic users prior to 11/4/2019 (when we rolled out the new architecture to production) can export their existing keys from the old architecture, if the previously custodial key management model concerns them. After old keys are exported, upon logging in again, new keys will be generated for users with the new delegated architecture. We will proactively reach out to the existing users and prompt them to migrate to the new implementation with improved trust and security.

Security Audit and Compliance

Fortmatic infrastructure has been thoroughly audited by NCC Group and A-LIGN and is the only SOC 2 compliant Web 3 compatible key management solution. All the controls and procedures required by the compliance have been implemented at Fortmatic. We also participate in white-hat hacker events and run a private program on HackerOne (if you are interested in joining the program, please send us a message to security@fortmatic.com with your H1 username). Fortmatic’s infrastructure also goes through vigorous penetration testing at least once a year.

If you find any security vulnerability, please report it to security@fortmatic.com. If you are not in our bounty program, we would love to invite you to join our program on HackerOne. Bounty will be awarded if it is confirmed a valid vulnerability.

CSRF and XSS

CSRF and XSS are all the common attack vectors for web applications. Fortmatic regularly audits 3rd-party scripts to enforce strict CSP headers to mitigate XSS and untrusted data ingestion exploits. CSRF tokens are used to ensure requests’ authenticity, which prevents users’ requests from being forged by malicious attackers. We adopt industry-best engineering practices building our products to make sure we are not vulnerable to any untrusted data injection. We are also rigorously implementing security controls and monitoring to further protect the end-users.

Phishing and Iframe

Fortmatic uses an iframe to provide dApps seamless user experiences, where dApps won’t lose users from having them to go back and forth between the dApp and wallet interfaces. To enforce our level of security, domain verification is required for any Fortmatic-integrated application that serves traffic on mainnet. With domain verification, the developer’s API keys will be blocked from accessing Fortmatic resources from all unverified domains. This is not just a one-time domain verification; the domain needs to remain verified for however long the application is using Fortmatic. Users are encouraged to set up multi-factor authentication (MFA) to further secure their accounts. Fortmatic also partners with 3rd-party services to defend against phishing and protect users from unauthorized accesses and operations.

SIM Hijacking and SIM Porting

SIM hijacking and SIM porting are well-known problems that mobile phone users are facing, and amazingly, the industry has raised awareness for this kind of attack. Fortmatic focuses on providing optionality for users with both email or phone login. Users can sign up with email or phone + password (only user would know) and add two-factor (e.g., Google Authenticator, Duo, etc.) for multi-factor authentication (MFA). This greatly improves the user account security and mitigates the concerns of SIM hijacking and SIM porting. Users of our multi-sig smart contract wallets will be even less susceptible to this, as transactions can be gated by the quorum or transaction limits.

End-to-end Encryption with TLS

Transport Layer Security (TLS) is the standard web protocol for encrypting data on the Internet. Web services provided over the Internet without TLS should be strongly discouraged. At Fortmatic, all service communication only supports TLS. No matter user data transported between Fortmatic servers and user browser is sensitive or not, it is end-to-end encrypted. This applies to the 3rd-party services that we made available to the end-users. But Fortmatic doesn’t stop there. We took it further and implemented TLS for our internal service communication. The communication between our servers and the internal application secret storage, Hashicorp Vault, is also encrypted by the TLS. In the unlikely event of Fortmatic servers being infiltrated, none of the sensitive secrets transported on the internal network is in plaintext.

By implementing TLS, it prevents man-in-the-middle attacks. Users using Fortmatic services are protected end-to-end.

Infrastructure Security Architecture

At Fortmatic, we adopt the industry-best standard to implement our infrastructure. From public/private network segregation to intrusion detection monitoring, we made sure we spent time to carefully architect and bootstrap the foundation of our infrastructure that is secure for our engineers to build on.

Network Setup

Fortmatic deploys services in virtual private clouds (VPC). Each environment (dev, stage, and prod) has a dedicated VPC, and no communication is allowed between VPCs except with the VPN’s VPC. Within a VPC, we split up our network into public subnets and private subnets. Public subnets are the only network with Internet access. Services deployed in public subnets are accessible to the world, whereas the ones in the private subnets aren’t. This allows us to deploy core business logic in the private network and protect those services from the Internet.

Intrusion Detection Setup

Fortmatic deploys intrusion detection systems in its cloud environment. We analyze all of our VPC logs, DNS logs and service logs to detect threats and any unauthorized access. We continuously monitor the traffic and take action when needed.

Data Encryption at Rest

Fortmatic encrypts all of its databases, snapshots, automated backups and replicas with the industry-standard AES-256 encryption algorithm. The encryption and decryption are done automatically when data is written and read from the data storage.

Why AWS?

There are many reasons why we selected AWS. It is by far one of the most durable, widely-adopted, and secure platform in the industry. At Fortmatic, customer trust is our top priority, and we need a cloud provider that can support us with that vision. We chose AWS because of its determination to ensure its platform is secure by implementing policies, controls, and procedures to monitor its services. Their effort aligns and supports Fortmatic’s vision to offer secure and reliable services to our customers.

It is very important to note when we are picking AWS services to use, we make sure the services are attested by SOC 2 and ISO. AWS has done a phenomenal job, including many of their services into the scope of their compliance programs. This is the level of commitment that we want to offer to our customers. And we hold the same standard when we are looking for our cloud vendors.

To further break it down, here are the main reasons why AWS is our choice: Security, Reliability, Availability, Scalability, and Optionality.

Security

AWS has put in tremendous effort to achieve thousands of global compliance requirements to show their determination in providing one of the most secure cloud platforms in the industry. Their security and data privacy controls have been regularly attested and validated against compliances. Their continuous innovation in the security space by offering new features to streamline and reduce the barriers to set up a world-class security infrastructure is well known in the industry.

Reliability, Availability, and Scalability

Once we have a solid foundation for security, we invest heavily in the stability of delivering our services to our customers. There are a few areas that we take into consideration with particular emphasis on three aspects: Reliability, Availability, and Scalability.

Being reliable means everything to Fortmatic. AWS offers excellent SLAs on all of its core services that we use. From the container service to the key management service, AWS offers more than 99.9% uptime. If we add redundancy to those services, we can even achieve higher reliability.

To ensure we have better reliability than it is specified in the SLAs, we need to achieve high availability. AWS offers 69 different availability zones and 22 regions around the world for us to deploy our services to. This allows us to deploy the same stack of infrastructure in different regions. The mirror setup will enable us to have high availability. In the event when a service goes down in a region due to natural disasters or power outages, we can route traffic to a backup region that is on standby immediately. The resilience of this feature is the core of Fortmatic’s business continuity plan.

Lastly, it is always a challenge when it comes to sustaining a sudden increase in service traffic (the example above is an excellent case). That is why being able to scale efficiently is very important. AWS provides numerous mechanisms to automatically scale infrastructure on the fly, depending on the current traffic volume. We can also voluntarily scale up and down as we see fit. Given the AWS global infrastructure setup, we can also deploy to different regions to better serve our end customers from the data centers that are closer to them. This can considerably improve network time and user experience. AWS also offers some cool features that allow us to split up database write-and-read nodes and scale read nodes as needed to accommodate our application requirements.

Optionality

AWS offers a wide variety of cloud services, which is the main differentiator when it comes to selecting cloud providers. It is AWS’s biggest advantage, which is also the important thing we see that we can benefit from in the future. We opt-in into their solutions after a series of evaluations. We thought about the vendor lock-in concern; we thought about the chances of AWS being evil and controlling our data, and we thought about all the service administrative work that we have to do if we don’t leverage their solutions. After all, we decided to trust AWS because they are a well-established company with many reputable organizations relying on their services. Also, we’ve seen the tremendous effort from AWS to build trust between their customers. Many services we leverage from AWS effectively remove the overhead from us needing to do any administrative work. For example, AWS RDS and AWS ElastiCache solve the two most significant pain points for a company when it comes to managing a database and a memory cache cluster. Productionalizing these two services presents challenges, not to mention the resources needed to maintain them. At a startup, we optimize resource allocation.

Conclusion

After all, we picked AWS for its properties of being highly reliable, scalable, and available. Google Cloud and Microsoft Azure are two other cloud providers that are worth considering. They also offer excellent security and commitment to make sure their platforms are secure. However, even if we’ve already codified our infrastructure in Terraform, switching between cloud providers while maintaining our services in the middle of the day is still challenging and resource consuming. That is why we wanted to start off with a provider that could support us with future growth. Given the Fortmatic’s need for its current and future products, Google Cloud and Microsoft Azure lack variety in service offerings compared to AWS. Nevertheless, we do not rule out the possibility for a hybrid cloud provider setup in the future to achieve a higher degree of service availability for our customers.

We are hiring!

Fortmatic is rapidly picking up traction and adoption amongst developers with many new projects actively in development. We are looking to hire a Software Engineer (DevOps/Infrastructure) and Software Engineer (Security) to go on the mission with us. You can also check out our other open roles at https://careers.fortmatic.com.

If you are interested in learning more about Fortmatic and integrating with us, make sure to join us in our Telegram group or tag us on Twitter!

--

--

Sean Li
Fortmatic

ceo @magic_labs @fortmatic | ex-@docker @kitematic | @uwaterloo alumni