Data Lake — Design For Better Architecture, Storage, Security & Data Governance

Best data lake recipe lies in holistic inclusion of architecture, security, network, storage and data governance.

I started writing this post with the intention to demonstrate a practical approach on how to build a data lake on AWS. However realized its important to first discuss some of the best architecture patterns before going deep with hand-on. I will be writing a few more posts on demonstrating a practical approach as well.

Note: As part of my continual effort to enhance the content, I would appreciate it if you could give your thoughts in the comment section.

Data-driven outcomes, forecasting, and predicting business trends is essential to any business. Today we see at least some sort of analytics in everything we do. Right from clicking on a website (Clickstream analytics), buying online (Customer behavior), Genetics, CRM, utilities, healthcare, and even elections we see analytics being used. Analytics is not anymore, a question to gain a business edge, but it has become essential to remain in business. It leads to organizations building Data Lakes or upgrading existing data warehouses.