Using Sketch and Asciidoc to Generate a Professional Tech Book
I wanted the interior of my upcoming book to look as good or even better than top of the line publishers. But the default look from asciidoctor was lacking. I then discovered the ultimate/unexpected typesetting tool: Sketch!
Sketch isn’t designed for printing, but the result is absolutely stunning when compared side by side to the output from asciidoctor. If you want to check out the full PDFs I have preview chapters at the end of the article and you can compare the results side by side.
First I’ll start with the end result so you’ll understand my motivation for doing this. I’ll then explain the process and tools I used to convert the original into the final result.
To give you a glimpse here are a few pages side by side:
The table of contents looks better. I could probably create a good looking ToC in asciidoctor by using styling but it would be harder to achieve that layout. I also didn’t really know what I wanted beforehand so the design was much easier after the fact than fiddling around with asciidoctor styling. But the big thing here is the layout!
Notice that my final ToC includes only two chapters, it doesn’t flow to the next page. The output from asciidoctor includes some portions of chapters 8 and 10 instead of listing complete chapters only. This is very important when you try to read a physical book, you don’t want to leaf back and forth to get the full picture.
The design of these pages looks very different!
At first glance you might think it just boils down to colors and some styles, since I didn’t know what I wanted when I generated the PDF those remained in the default form. I probably could have customized some of them through styling but not all e.g. the “this chapter covers” block etc.
Notice the chapter number on the right in my version, it “bleeds” to the end of the page. Bleeding in print means the color goes all the way to the edge of the page and leaves a mark on the side of the book to the side you’d be able to see all the chapter openings. I use bleeding a lot and mark each chapter at a different height as you will see in the next page. Bleed is pretty challenging as you need to get the layout “just right” in terms of size. One of the problems I had with asciidoctor-pdf relates to the page numbering on the bottom. It ends too low… But when I added padding KDP (Kindle Direct Publishing) wouldn’t accept the resulting PDF. I still don’t know why. I had to fix the height of the page numbering manually in sketch.
Also notice that there is a bullet missing in the original asciidoctor version. It moved to the next page despite the fact that there is room on the page. I arranged everything so all the bullets would fit on the first page.
Before I go into the code listing notice the chapter number marking the page on the top right. That means it’s a right side page. If the page was on the left it would have been on the other side. The number 2 chapter will be positioned lower and so forth. This allows you to hold the book to the side and instantly find the chapter you are looking for as the content bleeds.
Since this is one of the first samples it’s very heavily annotated and might seem intimidating. But the value of instantly seeing the annotation matching a line of code is immense!
Also notice the “Application Lifecycle Sidebar” I linked to after the 1st code listing. Notice that in my version it links to a specific page number which is pretty basic. Asciidoctor doesn’t do that as the process of page layout doesn’t all for two passes. This is something I was able to do manually as part of the process.
Also notice a minor recurrent bug in asciidoctor-pdf where the number 2 entry in the second listing got pushed to the next page without the number.
Notice that the #2 chapter marker is just beneath the height of the #1 chapter marker as I mentioned when discussing the bleed.
As you can see the page on the left is far more convenient to read. Image wrapping is pretty much essential.
I hope these images clarified my motivation for the doing the work I’ll explain in the rest of the article. Notice that this only applies to the print document. For the e-reader I chose to go with asciidoctor-epub3 which isn’t ideal but due to the problematic/fragmented distribution on e-readers I opted for the safe option. In the past I used the PDF conversion tool from Amazon and it limited some of the distribution options.
Sketch?
If you’re a developer you might be unfamiliar with Sketch. A year ago I was aware of it but didn’t bother learning it. I was used to Photoshop and since my day to day job isn’t graphics I didn’t feel the need to pick up a new skill…
When I needed to generate charts for the book with good looking text I needed a vector graphics tool. I tried many including open source tools, web based tools and Adobe Illustrator. Illustrator was pretty good but expensive and frankly I can’t stand Adobe… If you ever had to deal with Adobes customer support for billing issues you will know exactly why I can’t stand them. I ended up canceling a 4+ year subscription for creative cloud after I bought Sketch.
Sketch isn’t designed for printing. They are very clear about that. So I compared it to Illustrator which is designed for printing and found that Sketch was faster, easier to use and produced better results…
There is a caveat: Sketch doesn’t support CMYK. If color correct printing is important to you Sketch might be a non-starter. Since the book is black & white, and the color correctness wouldn’t matter that much anyway I didn’t have a problem.
Notice that Sketch is only available for Mac, it’s an amazing piece of software. I paid for it in full and I’m linking to a regular version, no affiliate or anything like that because I don’t want my recommendation to be tainted by perceived bias: http://sketchapp.com/
Asciidoc
I tried a lot of different tools when it comes to tech document authoring. They all have serious issues. Asciidoc is the closest I was able to get to professional printing. In fact it’s used by the biggest publishers in the industry, who produce gorgeous books.
But it seems that no one in the asciidoctor forum knows how they do that?
I’m guessing most professional publishers follow an approach similar to this one taken by Bob Nystrom. They probably generate a docbook XML file and import it into InDesign and effectively re-layout the book manually.
I tried working with Indesign and it was a pretty painful experience (regardless of my disdain of Adobe). I just couldn’t get the output from asciidoc/asciidoctor to format correctly for a week. There had to be a better way. Looking at the work Bob put into it (2 months worth!), I think my approach is potentially faster. Especially if I can automate it further…
I tried finding a professional typesetter or graphic designer who would be able to perform this task as a contractor. Unfortunately, while I found quite a few people in this field, I only found one who worked with asciidoc. I found her details after I was well underway with the approach I eventually took, so it was too late.
Asciidoctor-pdf generates relatively good looking documents by default. But as you saw from the images above they fall short when compared to the final result I wanted. Historically I used asciidoctor with the FOP backend that translates files via XSLT. I wrestled with it for days and wasn’t able to come close to something satisfactory.
Sketch and Asciidoctor-pdf
Finally, on a whim, I opened the resulting asciidoctor-pdf output file in Sketch. It took a couple of minutes to open initially but once it opened it was fast…
There were formatting problems with the resulting file so I tried illustrator which worked too but was even worse. I think some of the problems relate to the font choices and possibly some pixel conversion issues in Sketch.
Once I started working I had an epiphany. We can literally design every single page in the book as if it’s a graphic. Just like we design an app or the book cover. Furthermore, because I designed all the graphics using Sketch I could incorporate the original images!
This doesn’t seem important at first but it means I can replace raster (PNG) images with vector based sketch data. That means the graphics and fonts will be crisp regardless of the printing process. It also meant I can draw everything including the charts above, I can embed images everywhere and use complex concepts like bleed in my graphics.
Mistake and Recovery
Like all developers I wildly underestimate effort. 440 pages of work seemed like something I could breeze right through. Furthermore, Sketch has positioning issues for some texts which were very noticeable in the source code listings so the work was manual and painstaking.
It took me weeks of hard work to finally acknowledge that I was being stupid. I was thinking like a designer trying to do everything manually and didn’t automate anything.
One of the best things about Sketch is its support for plugins and open file format (it’s literally just a zip with json files within it). By the time I had realized that I had done too much work. Had I thought of it in the beginning I would have built a processor tool that would make the fixes to the json files automatically. That way I wouldn’t need to deal with the complexities of the plugin support. Unfortunately, it was too late for that so I wrote a simple plugin in CocoaScript that glued together the text within the selected block:
Forgive the horrible code, I’m not used to these tools. It took a lot of back and forth to get this going. But once I did all I had to do was select the code block and run this plugin… Then I could just drag every piece of text to the point I wanted it to be and draw the lines as they should be.
There’s one serious downside in this process. We can’t add/change pages in Sketch. Since page numbers are embedded into each page we can’t add a new page into the flow. We can’t remove a page either… This isn’t just page numbering, pages are asymmetric to fit the right/left positioning so any change might impact that.
That means that we can’t make serious changes that impact the page flow.
Another minor drawback is that links stop working the moment I edit them. I have no idea how to link within the document in Sketch. This isn’t a big deal for me. I use this only for printing and use a separate process to generate the e-reader version. But if you want to use this process for an online PDF it might be an issue.
About the Book and PDF Links
The book will be out August 16th. I recommend waiting for the print version which I’ll update on when it comes out. If you would like the digital version you can pre-order it here.
The first two chapters and appendices are available for free. For reference I also included them in the original PDF format for your benefit. I’d appreciate it if you send a link to the article and don’t redistribute the PDFs yourself.
This is the “good looking” output after I went over it with Sketch:
https://www.codenameone.com/files/uber-book/create-an-uberclone-in-7-days-free.pdf
This is how the raw asciidoctor-pdf file looks. Notice that it’s pretty good looking to begin with:
https://www.codenameone.com/files/uber-book/Creating-an-Uber-Clone-In-7-Days-before-sketch.pdf
