Software Engineering Your Data
Bringing the best of software engineering to Data engineering
Software engineering has been through a comprehensive evolution of framework, techniques, processes and standards over decades. Companies today employ mature practices for designing, developing and delivering software leveraging skills, technologies and processes. Software is not just about writing code and testing it but how fast you can go to market. In the world of SaaS and immense competition, writing clean code will not differentiate you. Still, your engineering practices, collaboration and cohesion within your organisation are some of the things that can put you on the path to success.
DevOps started in the late 2000s, taking the software industry by storm. A cultural shift that integrated development with operations into one continuous process, enabling rapid delivery, reliability, improved collaboration and a loop of feedback and iteration, has revolutionised software teams. Managing hardware to the cloud, waterfall to agile, manual testing to automated testing and many more advancements have brought about a mindset shift in the field of software.
The question is, how is this relevant to Data? Data is not a new topic; on the contrary, most of the software we use to store data in some form. However, since the early 2010s and the…
