Security Sucks! Part 2: The workforce
In years I was witnessing (and proudly took a part in) the full evolution of security model, from a growing startup to an established enterprise platform.
Previous episode part 1
The Great Recession hit Ukraine especially hard. All government spending was frozen, and we lost our only client, so I had to move on.
My next project was a very cool “business to business” system for “the workforce management”. Basically for every company with field technicians (like plumbers, and electrician ), delivery trucks, or salesmans etc.
In the end, the businesses of our clients were extremely diverse. Starting from small companies to huge enterprises like Telefonica and Dish. You can imagine how different requirements were. In years I was witnessing (and proudly took a part in) the full evolution of security model from a growing startup to an established enterprise platform, eventually acquired by a huge сorporation.
It started as any other small startup. The was a vision and few developers. The vision was, to make people happier by notifying them about the expected time of arrival of cable-guy, plumber etc. In the beginning, it was a simple customer notification system.
The security model of the first prototype could be very simple. One server per company and one password to access it. But obviously, any company has more than one employee. So we’ve added username and password to the database.
Now the boss from a client company wants to add new users but doesn’t want them to be allowed to add other users. So we marked some users as a “root” administrator.
Very shortly we found it difficult to manage individual permissions, so we created a “user type” so you can create the user of a pre-defined type. In this user type an admin can set checkbooks near permissions like “Allow activity deletion in Manage”.
The things went well, we grew fast, we got many new users with different business models. Now we need more flexibility in views and configurable operations with business entities like clients, appointments, tools . Guys were working hard, so now different users also can have a different visibility profiles so system can now hide or disable some input fields and buttons on views and editing forms. This actually created much confusion in the security model. In some cases, it was misused as a permission control. But not always data sent from the browser was checked on the server correctly, so even if user cant see the input form he could fake the request without any problems.
Another change was splitting the user and his current role (we call it resource) so some users can change their roles after the login. For example, yesterday I was fixing cables, today I’m on-call, and tomorrow I’m just a truck driver.
Somewhere at the same time, clients tired to manage the passwords and requested to use their corporate LDAP server instead of our database. “Ok,” we said, but turns out it was not that easy. You need VPN channel with each client and also all LDAP servers had different incompatible issues. I love you, Microsoft, you never leave developers without stupid work. So we end-up with the flag “LDAP server is MS Active Directory”.
We created several type of authentication (login policies) that admin can assign to the user.
So now it was something like role-based access control but “with the Blackjack and Hoo.. views”. As it was written in the docs “Each user type has a profile that defines security and display permissions, such as the user’s login method, the ability to use certain functions, and access to menu items and properties. They may also include custom screen context layouts.”
And the next client came with the new requirement “one department should not see orders for another department”. “Oh no!” we said,”But Okay” we said, and re-implemented all again to have groups and hierarchical tree for groups of groups.
But this not the end, “we need to go deeper”, we implemented “Organisations” as a higher order group. But even this is not the end,
later our sysadmins came to us crying and asking to implement multi-tenancy because at some point the number of individual system installations came to hundreds with dozens different supported versions. It was just un-maintainable anymore.
And we did it again, painfully with several fuckups in the way we re-implemented the security model to include check for the company permission first, the root flag, then the groups, then profiles, then views.
And another huge corporate client came and requested some certification where it was sad that it should be no “God-like” root access in the system.
We returned to our whiteboards again, carefully removing the root access flag and solving the “chicken and eggs problem”. How to create a user without escalating his access permission? We found some unique solution, the user type (role in other systems) includes two white lists “Can create users of the following user types” and “Can be created by users of following user types”.
And the next client came with the new requirement “we have our own corporate SAML Identity Provider instead of LDAP, and we don’t want to synchronize users and passwords with you”. “Okay,” we said, and again re-implemented all authentication again to integrate with any Identity providers SAML, OAuth, OpenID etc. This part was especially difficult and costly. It took more than half a year of my life and many other development and changes in almost all subsystems and the new central authentication server. We were heavily using our HTML5 mobile offline web application for field workers, and integrating it with SAML (that was never designed for this use case) required all our creativity and millions of tests. Plus, as it was with the LDAP, if you have a standard it doesn’t mean anything, all implementations of it will be different.
But as a side benefit of having a central authentication portal we got the one-page login for all companies and the two-factor authentication from Google.
This time I took a bit more responsibility and implemented more than was initially required, and it was right in time. The next client asked exactly this, two-factor authentication and OpenID from Google.
To summarize, It was my coolest project so far, the funniest team, an awesome life experience and a great learning. But because I was “familiar with this”(authz) I was always involved in all these reiterations of security models, it was exhausting for me. As I said, I hate working on security. It was one of the reasons to change something radically in my life.
So the learning from this part of my life:
Changing existing security model is an expensive and ungrateful job. If you look at the changing requirements above, you will see, they could be predicted. You will never invent anything new here.
If you care about the growth of your product, design and implement authz ahead of time, base on the well-known authorization models like Role Based Access Control (RBAC) on another. But implement it fully, don’t forget about the role tree hierarchy and don’t limit “access” operations by only “read/write”, make it extendable.
Follow current best practices for authentication and use single sign-on, like OAuth2 or OpenID or even better use existing IAM services like Auth0 or Amazon Cognito.
Authentication and Authorization model is not something you can do later. Based on all my experience(here and later), an agile iterative process in this area is hurting more than helping, it will cost you a lot-lot more than well-designed, even initially heavier, architecture with enough flexibility to model future use cases without too many changes and without aligning with the customers.
As soon as you have a second user Do it right the first time!
To be continued…
