Team BEIAI and how to make an AI webcomic/webtoon

I’m Luke Yang, BELIVVR’s CEO and a team that produces AI webcomic for LOOOL BEIAI is an AI webtoon content production team that produces and edits webcomic by utilizing stable diffusion rather than developing AI technologies such as SW development.

There’s a derogatory term for Korean AI creators, called clicker. It’s meaning is tey can’t draw, so they create by clicking. But our creator are all webtoon majors, and the reason they do AI webtoons is not because they can’t draw.

Dear BEIAI team members, please share your thoughts and introductions on the project.

1. Sun : This was my first experience with Stable Diffusion. It was interesting to work with AI and experience various things like, “Oh, I can do this, or I can’t do this. I mainly worked on making character and candidate selection.
2. Warnie : I dealt with AI for the first time, but it was easier and harder than I thought… It seems to be a special experience. AI is not as smart as I thought…^^ I mainly worked on conti, character extraction, and typesetting for translater.
3. Goofy : I think I had to work a lot because the character’s pose and expression didn’t come out as I thought. I mainly did conti, character extraction, and retouching.

Authoring Environment

We are using computer and Wacom One with Rtx4090. The rtx4090 graphics card used by the workers is expensive and accounts for most of the price of the PC.
I wanted to give you a new one, but I have a lot of Wacom ones that I bought while teaching webcomic at Billiver, so I am using that device. I will buy you a big Sintiq when I earn money.
In addition to this device, there are many small PCs with Rtx3090 in the server room, which are used to extract AI images for use in Billiver’s AI training platform.
Video memory is important for AI webtoon production, so I recommend the Rtx3090 and Rtx4090. If you need to extract high-resolution images or train your own models, I highly recommend it.

How to Create

We’ll be sharing more details in the future, but here’s a brief introduction to the process of creating a webcomic. There’s a lot of information scattered around the internet community, and there are some great tricks on YouTube.
However, all of these resources are not optimized for webcomic production, and we hope that more people can easily access AI webcomic production by referring to this method, so we will briefly share the method we have organized so far and hope that more people will approach AI webcomic production.
First, our production process starts with the assumption that we have the drawing contie for drawing of the webcomic. Currently, we are working with the outsourcing novel writer on the text contie, and internally, we are working on the drawing contie making and beyond.

Create a baseModel that fits the story moods of your webcomic

The great thing about stable diffusion is that there are a lot of models out there. https://civitai.com/ is a popular AI model sharing site where you can select the models that most closely resemble the webtoon you want to create.
You can choose just one, or you can choose many and mix them up. In our case, we wanted to create a unique model, so we mixed several models. The model we created will be used as the base model for the background and characters, which will set the moods for the entire piece.

Using Prompted Writing to Create Character Concept Art

In discussions with the team, we used the character’s traits as a hint to flesh out what he would look like. It’s especially important to capture the mood of the piece and represent the character well.

ex ) VRRI’s making prompt, a character for
<Drink this. You’ll go into space>

prompt : Purple Hair, girl, 1girl, Solo, Cute, Simple Background, White background, Beautiful eyes, Purple eyes, Beautiful skin, Clean skin, High Quality, Masterpiece, (Bobbed Hair:1.2), Natural Hair, Beautiful delicate eyes, old, K-pop style, Korean Webtoon Style, full bangs , long eyes to nose, Mature, Long face shape, Adult, Straight Hair, Short hair, Focus on face, Left tear dot, Dot under the left eye, Apron

negative : easynegative , ng_deepnegative_v1_75t, Long Hair, young, Curly Hair, Low Quality

Training Dreambooth Models for Character Creation

Once we have a set of prompts with the desired mood, we create multiple character faces, each cropped and upscaled to 512x512 images to train with Dreambooth.

Normalized images used to create the character vvri’s model

Normalized images used to create character berry models
Slide in the 512x512 normalized images, keyword the characters, and create a Dream Booth model. You can find a detailed training method for Dream Booth on YouTube. When creating a Dream Booth model, the default model is the base model created in the previous step to maintain consistency.
Once created, test the model with various keywords. At this time, when creating the Dreambooth character image, if you feel that the Dreambooth model is alone or overfitted, mix the base model a little more and fix the model for the character. This is a model that will be used over and over again, so make sure it’s good.

Create characters and backgrounds with ControlNet using a photos and 3D models in a Conti-like shape

Currently, we work based hand-drawn contie content and don’t draw characters and backgrounds all at once. because it’s a hassle with post-processing, but it’s easy to divide up the work process and it’s the same approach as the traditional webtoon production process.
We mainly use OpnePose to create poses by taking photos, or we use 3D characters to select poses and compositions, and use them to draw characters.
For the background, We set Background’s perspective drawing by SketchUp’s 3DModel background to create the composition and img2img to create the background Image.
The stable diffusion feature, ControlNet, which can be found by searching YouTube. so it’s a good idea to familiarize yourself with ControlNet as its use continues to grow.

Placing and post-processing cuts and speech bubbles

Placing cuts and speech bubbles is the same as in a typical webcomic creation process.
Shadow the composited image. We didn’t work with the background at the same time, SO AI image’s all area is very dense and detailed. This feature is bad for say important topic to reader in webcomic. so you’ll need to add shadows and light to make sure the background and characters don’t look out of place for important area strong. I’ll fix odd expression, and desaturate or Gaussian blur unimportant areas to give them more emphasis.

Fixing the look of a character’s outfit

You can fit clothes to some extent through LORa training, but LORA requires a lot of training data, so we haven’t tried to make LORA yet, and we’ve even tried to fix the output of the color of the clothes through the Cutoff plugin in stablediffusion.
We are planning to review the technology because it would be convenient to collect the uniforms made through the series and make them into LORA again.

Current Biggest Concerns

1. Create a system to increase the probability of producing the intended image
   We need to generate a lot of images to select the right characters and backgrounds, and in order to increase the probability, it seems that quality control of the self-generated model and management of workers’ prompts are important. Especially in the case of team work, it is important to share prompts, so it is necessary to manage them, and in turn, it is important to build a system to manage and utilize the best cuts and data.
2. Advanced workflow develop for character and background making
   The current method of drawing and pasting characters and backgrounds separately requires a lot of post-processing work, and it seems to lack naturalness due to the AI’s characteristic high quality of all areas, so we will consider changing to a method of drawing characters and backgrounds at once.

Closing remarks

While our approach may not be the best way to create AI webtoons, there is currently no easy way to create commercial-quality AI webtoons.
Most importantly, we wanted to show that it is possible to create AI webtoons in this way, and we hope that it will be helpful for those who want to use AI to create webtoons, and we will continue to share more tips and tricks as we discover them.

You can check out our AI webtoon on LOOOL.

LOOOL - Get premium AI generated webcomics site in VR