Retrospective: My Experience Writing Data Management at Scale
By Piethein Strengholt
After 1,5 years of writing and editing, I finished my book: Data Management at Scale! The book continues to build on my previous medium posts: ABN AMRO’s Data Integration Architecture, ABN AMRO’s Service Oriented Architecture 2.0, ABN AMRO’s Streaming Architecture, and Misinterpretations of Microservices and what Microservices truly is about. It provides a perspective on data management and data integration. It provides tons of blueprints, principles, best practices and gives you a deep dive into disciplines like data integration, analytics, governance, data security, data quality, master data management, and metadata management. At the same it keeps things practical, so you’ll find patterns to quickly get you up to speed.
A large number of people have asked me about the amount of time, effort, and what it is like working with a publisher. In this blog, I share my experience with you.
Sneak peek of the book
The book provides a perspective on data management and data integration. The book is infused with my personal experience, driving the data architecture agenda for a large enterprise as a chief data architect. Executing that role showed me the huge impact a good data strategy can have on a large organization. I wanted to share this strategy, but also explain how to engineer and execute this. Because many of the data management areas and data integration aspects are heavily intertwined, I decided to build on the vision slowly. Starting with the core disciplines that define data management, reviewing the overall architecture, and zooming in on various areas.
Why I chose O’Reilly over self-publishing
I get a lot of questions about why O’Reilly and what the process has been like working with them. O’Reilly is a learning company that publishes books, produces tech companies, and provides an online learning platform. They are the number one tech publisher. In general, their books are of good quality. Also, I like the original detail that each book has a unique animal on the cover. Next to that, I have been learning from O’Reilly since I started programming as a kid. So, this in combination with O’Reilly being an industry leader, is the main reason why I chose them. Being published on O’Reilly also helps with my credibility and reputation as a data expert.
Another reason for choosing O’Reilly is because of the way they facilitate the writing process. If you self-publish, you need to do everything yourself. To name a few; find an editor, print the book, distribute it, and so on. O’Reilly is known for having a really great process. After experiencing it myself, I can only affirm this! Let me explain it some more.
For writing and formatting a book, O’Reilly uses Atlas: A Git-based, web-based platform for publishing and collaboration. As an author, I use the markup language to write and style my book and then push any intermediate results into a repository. Editors read and review my changes. This made the process very easy and efficient. In addition, both authors and editors can choose to “generate” a book — in its current state- to PDF, EPUB, and other formats. See what it looks like below:
About the process
Writing a book was fun, but also one of the most difficult things I’ve ever done. O’Reilly has a real passion and sets high-quality standards. During the onboarding stage, we made an agreement about the amount of content, structure, and number of chapters that must go in. This means that you must be very disciplined. I set a goal for myself to complete one chapter with high-quality content a month. On the weekends, I wrote for four to six hours. During the days, mostly nights, I wrote two to three hours a day, at a minimum of three days a week. During this time, I also spend a lot of time reading and studying other materials.
The first time I shared something with my personal editor was after drafting three chapters. I remember just seeing red lines and hundreds of corrections and remarks. The feedback was very helpful, but also direct. So, I re-wrote everything, while still adhering to the overall planning. We had the same process for every new chapter.
After all chapters were completed, the initial book was shared with several technical experts. Again, I received a ton of feedback. For instance, one person criticized the bias I had against data warehouses. Also, they thought the introduction was too long. Taking this feedback to heart, we had discussions and decided to drop the first four chapters and re-write them into one. I must admit this greatly improved the overall quality of the book.
After making all necessary changes the book was handed over to the production team. O’Reilly uses a default workflow, click here to view. During the production process, many people are involved. Also, the schedule is very tight and as the author, you have less control. O’Reilly takes over your book and starts editing. You mostly watch and answer queries. During this stage also the image team is consulted. For me, that meant re-drawing most of my images. Personally, I am amazed at how many persons were inquired to help and create the final book.
Lessons learned
The most important lesson I learned while writing, is creating a good structure. Working as an enterprise architect means that I also structured the book. This good structure prevented moving content between chapters when carrying out changes. Another lesson is to carry a notebook and pen around during the day. This prevents you from losing good thoughts or insights while staying focused on my job at ABN AMRO.
If you are considering writing a book — just go for it! Writing a book is an inspiring journey. It will not make you rich, but I found that it made me a better person. Do it to sell yourself.
Reviews
It is really great to see that most readers enjoy reading my book. For example, Valdas Maksimavičius says it’s a brilliant book about Data Management. Well structured, covering data platform design aspects, integration patterns, and wider data governance nuances. Michel Lutz calls it a must-read for every Chief Data Officer.
Do you like this article, and are you interested to see the results? You can order the book here.
