Data Lifecycle management
Just a cat
Long ago, the Nasca people drew this cat on a rock near Nazca in Peru. Together with drawings of other objects, it is data, which has been saved forever.
How can we be sure that this data is real? That it wasn’t changed by time? That it was meant to be like this and not scheduled for modification? Was it finished or incomplete? Is there another copy of it?
As you can see, there are a lot of questions about the data itself. Unfortunately the same kind of questions can arise on data in current systems. Somehow, it seems to be right but we cannot be sure. This is a problem, because we need to be 100% sure; we expect that from the system.
We have to find a way to assure quality of this data, possibly by making sure it is imported correctly (validations etc.) but more importantly when it is already in the database. Changes by users, software, data structures, or business logic: it all could impact the data. But if it did: how do we guarantee it is correct?
‘Lifecycle management’ is the term I use for all automated measures to accomplish this.
I did not create the cat
I am sure I am much older than most (probably all) of you. I took my first data management courses in the 80’s. So much of the stuff I learned is outdated. But data quality is not outdated at all: in fact it is more important than ever for reasons such as compliance, audit-ability, and privacy.
Nowadays development goes fast. Agile streams working in sprints cause developers to deliver functionality rapidly. In many situations data correctness is or seems to be less important than speed: the most important part of a solution is developed immediately while the rest of the solution is put somewhere on the product backlog.
I have investigated this a number of times. I found that creating a ‘quick-win’ solution costs about one third of effort of creating the ‘first time right’ solution. But creating the ‘first time right’ solution takes about one third of the effort of creating a ‘quick-win’ solution and then also having to finish it later while also solving all the problems that were caused between those two.
Shocking, isn’t it? This is assuming that repairing all data and history is even possible at all.
Another discovery is that the top-5 time-consuming maintenance issues were because of system inconsistencies three years after implementation which were caused in the first few months after implementation, because lifecycle management was not in place or incorrectly implemented.
More findings on the cat
Having Mendix (citizen) development experience for about 10 years now, I ran through a number of my applications. I found that about one third of my microflows and database calls had to do with data quality/lifecycle management. I suppose this is valid for most Mendix applications.
So rapid development can be 1/3 faster if lifecycle management is included in the platform. It’s my hope that Mendix considers, decides on and implement lifecycle management aspects in its platform in the future.
A cat forever
Until we get an official solution in the future, I will go through these aspects (>20) in blogs following this one. I will describe the solutions I implemented, and how they could be implemented in the Mendix platform or in your application.
I hope to get feedback from you! Maybe these aspects are already covered or there are already apps or modules available for them. If not, we can enforce data integrity to guarantee its own quality from now on.
Do you like the idea of addressing lifecycle issues and solving them once? Then your road to automated lifecycle management has begun. After all, from a user and system perspective, we need to be sure the cat is still there in 100 years, and when we look at it, we know it is a cat.
Read more
From the Publisher -
If you enjoyed this article you can find more like it at our Medium page or at our own Community blog site.
For the makers looking to get started, you can sign up for a free account, and get instant access to learning with our Academy.
Interested in getting more involved with our community? You can join us in our Slack community channel or for those who want to be more involved, look into joining one of our Meet ups.
