How I wrote, illustrated and published 3 children’s books assisted by AI.

Sre Chakra Yeddula

Published in

Generative AI

14 min readFeb 13, 2023

Follow my January 2023 journey of my yearlong project or follow my website.

Introduction:

Writing and illustrating children’s books is a creative and fulfilling task, but it can also be time-consuming and challenging. With the advancements in technology, we can now use generative AI tools to help us in this process. These tools can assist with writing the text, creating illustrations, and even generating the entire book! In this blog post, we’ll explore the process and benefits of using generative AI tools and some of the most popular options available. We will also cover the process it takes to get your book from your brain to a bookshelf!

For a TLDR version of a video , Skip to below video

Here is a look at the entire process

Step 1 — The Writing Process

Step 2 — The Illustration Process

Step 3 — Putting it together for eBook

Step 4 — Putting it together for Physical copy

Step 5 — Publishing

Process flow for using AI to publish book

The Writing Process:

To get started, you’ll need to choose a generative AI tool that meets your needs. There are many options to choose from, including online generators and software programs. Here are some options for you that I have tried as part of this journey.

OpenAI’s GPT-3 Playground: With GPT-3 Playground, you can input a prompt or outline for your story, and the tool will generate the text for you. For example, you might input a prompt like “Once upon a time, there was a little girl named Sally who lived in a magical forest. She loved to explore and have adventures.” The GPT-3 Playground will then generate a full story based on your prompt.

OpenAI’s Dall-E: Dall-E is another AI tool that can generate images from text prompts. Simply input a description of what you want your image to be, and the software will create it for you. For example, if you type “A knight riding a dragon while holding a sword” into Dall-E, it will generate an image of a knight on a dragon wielding a sword.

Midjourney: Midjourney is a tool that helps you write and illustrate your children’s book. With Midjourney, you can input your story text and specify the style you want for your illustrations. The tool will then generate a full book for you, including both the text and illustrations. For example, you might input your story text and specify that you want your illustrations to have a colorful, cartoon-style. Midjourney will then generate a full book with both text and illustrations based on your specifications.

AI Dungeon: AI Dungeon is another tool that allows you to generate text based on your inputs. With AI Dungeon, you can specify the setting, characters, and events in your story, and the tool will generate the text for you. For example, you might input “Sally is in a magical forest, and she encounters a dragon. She must find a way to defeat the dragon and save the forest.” AI Dungeon will then generate a full story based on your inputs.

Artbreeder: Artbreeder is another tool that can help you create illustrations for your children’s book. With Artbreeder, you can specify the style, color palette, and mood you want for your illustrations, and the tool will generate them for you. For example, you might specify that you want your illustrations to have a soft, dreamy style with a pastel color palette. Artbreeder will then generate illustrations based on your specifications.

These are just a few examples of how you can use generative AI tools in the process of writing and illustrating a children’s book. By using these tools, you can save time and effort and still achieve professional-looking results.

The options are plentiful, so it’s important you pick the tool that fits your purpose. Here is how I settled on my choices. I essentially boiled the choices down to two categories.

AI Text generators:

These are tools that generate text from a large corpus of books. They use natural language processing, machine learning, and artificial intelligence to produce text based on the given prompt.

I chose GPT-3 as it offers great flexibility in generating relevant results with few parameters and has a large training set.

AI Image Generators:

These are tools that generate images from text prompts. It uses neural networks to generate realistic-looking images based on the given prompt.

I chose Midjourney and Dall-E as they had a comprehensive set of features, offer multiple image formats and provide an easy-to-use API. I could not choose between the 2 so I chose them both and split their use between 2 books. This way I will be able to offer a good comparison between the 2.

Once you have chosen your tool, you need to come up with the general idea for a book. I have found that relying on the generators to generate an idea for a book is produced rather uninspiring results. See below some of the rejected ideas that ChatGPT generated that clearly warrant writing a whole book about. Remember that these tools are trained on already written works and hence will find it really difficult to come up with a completely original never done before concept. Their best quality is to assist you in producing content once you have the idea. This is the secret sauce of producing good content.

Once you have your idea, the next step is to produce content. At this point, do not worry about the quality of the content. This was my biggest problem for me in completing these projects prior to AI’s help. I struggled to get the words on the page. This is where AI excels, it can quickly generate large amount of content from a few prompts.

Once you have the initial copy, then it’s time to refine and tweak it for maximum impact. AI technology is pretty good at producing generic content but not as great at creating personalized and nuanced output that resonates with readers. This is where your own work comes in. You need to tweak it to your unique voice, style and message. The gotcha from before still stands true here. AI is good at producing works based on the past, you need to produce content that is new and unique.

I will describe this process and my experience in great detail below:

Once i had a general idea, I sent the prompts over to ChatGPT and produced content. This took a lot of tries to get the prompting right. Below are a few examples of the prompts that I used successfully and some that I had to reject.

Successful prompts:

- Give me a children's Story for a 4 year told that That thematically
uses fruits and equates them to feelings In the about story can you
also give the strawberry the unique personality of being able to
understand feelings.

- Can you also have these strawberry teaching the other fruits it's
strength

- Write me a bio about a super hero team called math squad with
 the members being plus-man, minus-girl, multiply and another 
one of a dog superhero

- Can you add some puns into this text.

Rejected prompts:

- What can a title be if this was a children's book

- Convert the above into a script

- Introduce three fruits and give them unique personalities

- In the about story can you also give the strawberry Is an emotional
characteristic That Comes out

- Make the above into a 1000 word story for a 4 year old with exciting
and pun filled dialogue

- Can you generate a catchy title for this childrens story

You can see a pattern building here. Its subtle and cannot be explained but prompting is an art form on its own. See below for a good breakdown of a successful prompt both for a text generator and one for an image generator.

Once you get the hang of it, I found that the speed with which I was able to produce quality prompts got faster. At this point the whole story was just that a big blob of story content. This is where the human element comes in. I took the story and applied basic principles of splitting the story up into chunks based on experience (I have a 3 and a 4-year-old to whom I read these kinds of books daily so I have a hang of what good page structure needs to look like). For people that are unfamiliar with the process. There are great resources on Upwork and Fiverr that you can hire for a quick turnaround.

The Illustrating Process:

Once you have gone about splitting your story. The artistic side of you is now on call! The first step here is to storyboard the story. This is important because it gives you a good idea of page structure. The prompts you need for the art generator (more on that below) and the size of the book (Click here for my recommendations on size and structure of the book). Below is an example of my storyboards. As you can see there are a lot of edits to page structure and how it needs to be laid out. Storyboarding is also where you can create a rough sketch of what you want illustrated. Now in the past this would have been a challenging step for people that have a tough time illustrating or sketching. Not anymore! you can just write in text what you want your sketch to look like and then use the image prompters to either sketch it or fully illustrate it for you.

Here is my storyboard for one of my books.

Image prompting:

Once you are done storyboarding, the next step is to start illustrating with the help of image prompters. This is the toughest part of the whole process. Before we get started let me do my rant on this subject.

Image prompters or Text-to-Image generators are the most inconsistent of all the generative AI tools out there. No matter how good your prompting is they have major drawbacks. Some of these are the limited choices of stock art, the lack of color and detail, the inconsistent results they generate, and their inability to provide any sort of coherent results from one prompt to another. They also do NOT do well with faces.

The above drawback is important to understand because this determines your art style and the kind of output you are willing to produce and accept. This wasn’t that big of a deal for me personally since I have the ability to take a sketch and modify and make my own illustration, but I can see how this can be a stumbling block for many. I would recommend for these folks to take some additional help on Upwork and Fiverr to finish or to finetune the artwork.

See below some examples of me using Sketchbook on my tablet to further illustrate the images from the image generators.

Using tools like Sketchbook to further illustrate

how to produce quality Images

I want to describe my experience and provide my recommendations on how to produce quality Images.

First, pick an art style that does not need a lot of detail. for example, I picked a layered paper style for one book and an illustration style for the other. do not pick photorealistic or very detailed character pieces. That will derail your journey real fast.

Secondly, try to separate your image into background, foreground and characters. This allows the AI engine to produce more consistent images. For example, In my books, i wanted to do a burning city as a background with robots in it. The first thing I did was generate the city by itself. This allowed me to get a lot more background images of burning city since the AI did not have to account for robots or characters. It kept the results consistent as well. Create your foreground or character images separately. I will talk about how you can put all of this together in below sections.

lastly, try to save your prompts and use the parameters. Parameters like aspect ratio, stylize greatly enhance your experience. In the Dall-E engine use the image editor to its most potent capacity.

Here is a good cheat-sheet for you to get cracking.

Midjourney Vs Dall-E:

In my experience I ended up using Dall-E for more artistic work. It was really good at producing art type images that looked like art and not computer generated. It did struggle with detail though. Midjourney on the other hand was able to generate good detailed illustrations and images but you could easily tell that it was a digital type art or AI generated. It lacked the finesse and the layering that real art would have.

Putting it all together.

Before we get into this section, i am going to sidebar here and ask you to go through my book size section here. Come back once you have finished reading that, understanding book guidelines is key to produce quality content.

Back? alright so now that you have your text and your images. Its time to put it all together. There are lots of great options for this part of the journey that are not AI tools. Some examples are

Adobe Photoshop : A powerful image editor that has lots of features to help you with the composition of your images and texts.

Adobe InDesign: A versatile layout program that can be used for a variety of projects including books, posters and more. It’s great for page layouts and interactive documents.

Canva: A web-based design tool that makes it easy to create images, videos and presentations. It’s perfect for those who don’t have the time or resources to learn how to use Adobe products.

Inkscape: A vector graphics editor that can be used to create images and illustrations for books.

GIMP: A free graphics editor that is also open source. It’s great for producing high quality images quickly and easily.

All of these tools will help you get creative with your content and make sure everything is put together in a professional way.

My favorite of all of the above was Canva. Its simple web interface and good feature set made it the perfect choice for projects like mine. It allowed me to quickly and easily create professional looking book layouts. It also allows me to take my work from my laptop to my tablet and even my mobile phone.

Protip: Spring for the pro version because of tools like “Image background remover”, more images in templates etc., These are a godsend when you are editing Images generated by AI.

Here are a few screenshots of my work in progress using Canva.

Now that you have gotten your book all designed it’s time for the big kahuna!

The Publishing process:

The publishing process simplified is in 2 categories, publish with a publisher or self-publish. What no-one tells you is that self-publishing is so easy even the most novice, rookie, new to this whole thing person like me was able to do it in 1 day. There are some gotchas though and I will cover them below.

Here is a good video that I used to understand the self-publishing process.

Here is another one talking about book trim sizes and guidelines.

The first is to understand what are the popular available options. Amazon KDP and Ingram Spark are the big hitters. For the sake of simplicity, I picked Amazon KDP.

The interface to self-publish on Amazon KDP is very intuitive and can be used without much prior experience. Some gotchas include.

Use the kindle book creator to make your kindle e-book. This simplifies the process a lot.

Use the Amazons inbuilt ISBN generator to simplify the barcodes that are needed on every book. Pro tip: Kindle books don’t need these, paperbacks and hardcovers are the only ones that do. Only downside here is getting an ISBN generated on amazon means you have to get a new one if you publish outside of Amazon’s ecosystem.

Gotchas of publishing on KDP:

Pay attention to trim sizes, Safe spaces and margins.
Use the Amazon KDP trim size calculator to design your Canva or other
tool’s page layout. 8X11 is not exactly that!
Order proofs in different finishes to actually see how it looks,
Matte Vs Glossy, Premium paper Vs standard paper.

Once done. Order your proofs and use AI tools like Jasper.AI etc., to write your summary and descriptions. Then Publish away!

Book and Trim sizes:

Book and Trim size setting is very important. Use the KDP calculator here to make sure you have the right setting.

Here is another one talking about book trim sizes and guidelines.

Pitfalls to be aware of:

While generative AI tools can be incredibly helpful, there are a few pitfalls you should be aware of. One of the biggest challenges is that the AI algorithms are not always perfect and can sometimes produce unexpected results. You may find that some of your text is misinterpreted or that the illustrations are not exactly what you had in mind.

Licensing Considerations:

Another important issue to consider is licensing. Some of these generative AI tools come with restrictions on how you can use the content you create. For example, depending on whether you paid for the service you may only be allowed to use the book for personal use and not for commercial purposes. It is essential to understand the licensing agreements for the tool you choose to ensure that you are using it legally.

Controversy over fair use:

Now this is an important topic for me to address.

Do works produced through generative AI’s not give enough monetary and name contributions to artists and authors whose works are being used for the underlying dataset?

The answer to that question according to me is Yes. Although the art or work being produced is unique it has used some baseline data to come up with it. I would posit that the least the tools can do is cite references and sources for the content. For the monetary compensation part. I don’t have a good answer, We will have to wait and see how this landscape plays out.

Conclusion:

In conclusion, generative AI tools offer a fantastic opportunity to help us write and illustrate children’s books. Whether you are an experienced author or just starting out, these tools can help you bring your stories to life in a fast and convenient way. With the benefits of using AI, the process of creating a book has become easier and more accessible. If you’re interested in exploring this new technology, give one of these tools a try and see what kind of amazing results you can achieve!