Migrating to Amazon Cognito
This post is part of the Digital Transformation series, for better reading experience I suggest to read the “foundational article” first here: From Monolith to Autonomous Services and Teams.
If you are coming from a home-grown User and Authentication solution, moving to AWS Cognito makes a lot of sense. It’s serverless, super scalable, and managed service so it will free you up from maintaining the service. Furthermore, keeping up with security standards and multitude of attack vectors are mounting tasks. AWS Cognito is super cheap (Free for up to 50K Monthly Active Users) and it’s the get go Security service when using AWS Amplify, another cool service for building modern apps.
There will be 4 key activities in the migration:
- User migration
- Access Control Migration
- Switching Authentication to use AWS Cognito
- Develop Admin UI to manage Users and Permissions
In the final part of this post, we will discuss the process in the context of Event-First Architecture.
User Migrations
User migration has been widely covered in the past, and generally the strategy fall into one of (1) One Shot migration or, (2) Slow migration
One shot migration can be done via user data import to Cognito, as described in this blog: https://aws.amazon.com/blogs/mobile/migrating-users-to-amazon-cognito-user-pools/. In this approach, the user password will get reset and users will need to enter new password. The slow migration takes the less intrusive approach, since users can use the existing password to login to AWS Cognito. This is done via Lambda trigger, as explained here: https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-import-using-lambda.html.
Access Control Migration
This is the topic thats not been covered extensively. Your applications most likely have complex access control scheme, and each application may vary in the way it authorises the request. We will explore how AWS Cognito setup can make the transition less painful.
The gap between what AWS Cognito can provide vs what the current permissions system requires can be described below:
In Cognito, User can be a member of one or more Groups, and that’s it. But current systems have complex permission model. The current system implements RBAC or Role Based Access Control as described in User-Role relationship. In addition, the system provides Row Level Security (RLS) as described by the User-Access Plan relationship. Some sensitive contents are assigned to Access Plans, and users who are granted access to the content are required to be part of Access Plan. Furthermore, an Access Plan is defined within an Organisation / Division context in the company.
To accommodate these requirements, we need to fit various membership contexts into Cognito Groups. We can use Group prefix for different grouping context as follows:
With these naming conventions, a Role called Admin will be stored as R:Admin in Cognito. Marketing Division will be stored as O:Marketing, and HR Confidential Access Plan will be stored as A:HR Confidential. There is other option, which is to store ID as Cognito Group name, and human readable name as a Group Description. It’s entirely your choice.
Pre-Token Trigger action is a mechanism to alter JWT claims generated by Cognito, and it’s optional for our need. The default claim generated by Cognito will simply specify the group name in the token. This makes the token harder to understand since it is internal IDP naming convention. To decode the claim to use the legacy system vocabulary (organisation, access plan, role), Pre-Token Trigger can be implemented as a Lambda function. So instead of claims:
It will be transformed to
This is optional though, as the information will be used by the new Authorisation logic that yet to be developed at this stage.
We have covered the permission mapping part. Next is how we can push individual user permissions to AWS Cognito. Well, it will be different approach for different User Migration strategy. In One-shot migration, the user permissions need to be updated immediately after all users are imported. This will need to be done by custom script that update user group membership via Cognito Admin API. In Slow migration approach via Lambda Trigger, the same lambda trigger can include the logic to update user group membership.
Switching Authentication to use AWS Cognito
We need to update our applications to use Authentication provided by AWS Cognito. Using OIDC is the natural choice, since OIDC is a standard that is supported by libraries in many different languages. OIDC is so pervasive that your framework may provide integration with the internal permission system. We will create new OIDC Client in AWS Cognito for each of our existing apps, and then update our app to use the client.
Develop Admin UI to manage users and permission
AWS Cognito does not provide Admin UI beyond what is provided by AWS Console. You will need to develop new Admin UI for your end users. The Admin module will use Cognito Admin API to update users and group memberships. I will cover how we can rollout out Admin UI at the end of this post.
Migration in Event-First Architecture
Event First Architecture will give you more options how you perform the migration. The following diagram shows how the migration is done in our architecture. All components in this diagram are taken from earlier post about Autonomous Services, except the IDP (Identity Provider) Service.
IDP Service is a new Autonomous Service that is responsible for User directory and Authentication service. Amazon Cognito is at the heart of the service and it follows a typical Event Sourcing pattern. The IDP subscribes to the events mainly from the part of Legacy App that provided the identity service. The event consists of user data, group and permissions. The event stream flows through via Event Hub. The listener then materialises the incoming streams into Cognito users, groups and user-group assignments.
Finally the SAGA pattern can be implemented to guarantee eventual consistency. When the Admin UI creates or updates records, it will emit events to Event Hub that are subsequently consumed by the IDP listener. The listener then translates the event into User pool records. When IDP fails to update the records, the IDP will generate a Failure event that can be used as indication by upstream service to rollback the data changes.
If the user reset password is acceptable (as in One shot migration approach), Event-first architecture will give us more options below:
- Allow each app to migrate at it own pace, since the old authentication service continue to work side by side with new authentication system. We can also rollout the less risk app first, and then perform migration for the more mission critical later on. This will reduce overall project risks
- Use the existing Admin UI to manage user creation and permission. Redeveloping Admin UI is a major task, without Event-first migration you will need to complete this part as part of the migration activities. This may increase project timeline and risk
Event-first architecture can reduce overall project risk and project timeline
Develop new Admin UI to manage users and permissions
The existing Admin UI may work well so far, but probably we want to have better UX, and lower maintenance cost in the long run by adopting SPA with modern frameworks.
The first decision we need to make is the storage. The most sensible approach for our situation is to use the existing user and permission database. Since it’s a relational database, naturally would be a good fit for the Admin module and requires no external Index like ElasticSearch. It saves us from complexity and extra cost. If we use the exact same databases being used by Legacy App, we do not need any migration or data sync. During cut over, we just need to disable the old system to prevent data integrity issues.
There are numerous choices on how to implement SPA with BFF (Backend for Front End) Service. Admin UI has a standard user experience where the user is presented with a List View of the data, along with a Create, Edit and Delete button on the same screen. To help users navigate, we provide Filter and Search. The Admin Module is so standard that we can employ a Higher level framework to do the job, to save us time, and streamline how we write code. Some of the most popular solutions in this space include Retool, Admin Bro, React Admin and Laravel Nova.
A new framework called Refine, an Admin UI framework inspired by React Admin — can be a good option. Unlike React Admin, Refine provides a stunning UI right off the bat, based on hugely popular Ant Design
Since the CRUD functions are so standard, with Refine you can complete the whole administration functions within days. Users, Roles, Organisations, AccessPlans, etc can be defined as Resources with a common CRUD screens that can easily be created.
Since we already have the database in the backend, we can just roll out the backend that is already supported by Refine: https://refine.dev/docs/core/providers/data-provider, so we do not have to develop from scratch.
One good candidate for backend is NestJS, the most popular enterprise grade backend framework, and it’s supported by Refine team internally (see NestJS CRUD Data Provider )
Unfortunately there is no ready Auth Provider for OIDC/Cognito, so I decided to create one and make it public so you can use it in your own project. Go over to Github repo here: OIDC/Cognito Auth Provider
That’s it reader. Hope you enjoy my post and found it useful for your own Modernisation project. Don‘’t forget to share and give some claps :)
