Subcutanean: Generating the Final Books
After a long development process, earlier today my custom Python script finally generated and rendered the first Subcutanean that will be printed and delivered to a customer. Here’s the moment in all its Terminal glory:
What’s going on here? Kind of a lot. Let’s break it down, shall we? (Or if you need more context first, here’s a gentler introduction to the Subcutanean project.)
This is a custom book for a Singular Subcutanean backer (supporters at this level pledged for a book that included a unique piece of content based on a prompt they provided). In the command at the top, I’m setting the output format to PDF, and using the --set option to manually turn on the particular bit of prose written for this backer. The --gen option sets the “seed generation” for this render, which is basically a fancy way of indicating “the first digit of the seed.” All Indiegogo backers will receive Generation 1 books, meaning their seed’s first digit will be 1; after fulfillment that generation will be retired and no subsequent books will ever again have a seed beginning with that digit.
Each seed generation has its own persistent counter to ensure no book is ever regenerated with the same seed. At the top you can see it retrieving the next seed and getting 10012. (Yes, there were 11 screw-ups and tests before everything was 100% ready to go. You don’t want to know what the counter’s at for the seed generation I designated for testing…)
You can then see the book from seed 10012 being generated twice, the first time marked “prelim.” Why twice? It’s to do with the End Matter items and LaTeX PDF formatting. You might recall one of the campaign’s stretch goals was unlocking different pieces of “End Matter” that can optionally appear at the end of the book. Because keeping each print book a fixed number of pages makes the logistics easier at the printer, books that are shorter than the maximum length have extra space, which can be filled up with things like an About the Author page, Backer Thanks, alternate scenes, and so on.
(The max length is actually 239 pages, by the way, which is the length if every single variant text uses the longest possible version… pretty statistically unlikely but possible. The also-unlikely shortest possible version is 209 pages. Because of how randomness and bell curves work, the vast majority of books end up in the range of 221–227 pages; the rare seeds producing books longer than 232 pages, the fixed size I decided to use at my printer’s, are skipped over if the target output is for print.)
To fill the extra space with End Matter, though, I need to know how many pages I have to play with after the book’s been completely generated, typeset, and rendered… and by far the easiest way to do that is to just do the whole thing once without the End Matter, check how many pages there were, then do it again with the addition of End Matter bits you know will fit. That’s what you see going on in the screenshot.
For each pass, the system reads an input of raw .quant files to parse — the file “full-book-manifest.txt” lists each chapter file in order. (During authoring and testing, it was often much quicker to just render the particular chapter I was working on.) The bit about “confirms” ties to my system for validating variant texts: in this case, no text has changed, so nothing needs to be re-confirmed.
Collapser then renders out a version of the whole book unique to the given seed, and does it pretty quietly if there are no errors. By the time the “vars” line you see next shows up, it’s all done: this is showing how all the named variables got set for this rendering. (There are also several hundred more unnamed variables for smaller variation throughout.)
Collapser then turns this version of the book over to the renderer, which in this case is for PDF output. This first converts the raw text with its generic markup to LaTeX, including adding a bunch of setup at the beginning with print specifications, the title page, and so on: then triggers a command line LaTeX compiler to render the generated .tex file to PDF. Finally, a different program is called that can get statistics on a PDF, and it checks for a wide range of possible output errors or unexpected results, and also returns a page count.
In the final steps of the “prelim” phase, you can see the program checking how many extra pages it has to work with (in this case, for a 225 page version, 7) and selecting a few End Matter items to fill up that space. The whole process then happens again, but this time with those End Matter items manually appended in. At the very bottom you can see a 231 page PDF has been rendered: the last step is to add 1 page of padding to reach the desired page count of 232.
Whew! Each one of these steps has been a journey to get right. It’s all running smooth as clockwork right now, although remind me to never upgrade anything on this laptop ever again.
Get your own unique copy of Subcutanean, or subscribe to my project mailing list for infrequent announcements of my new and upcoming projects.
[Index to all Subcutanean design posts]
