Using Synthetic Data for Dermatological Suggestion Models with Apple’s CoreML

DALL-E shows immense potential in enabling the rapid development of new AI models.

Abstract

The following paper explores the generation of a CoreML model using Apple’s CreateML framework to enable the rapid conceptualisation and implementation of a suggestive dermatology model to support acne patients in understanding the severity of their acne and to recommend the best courses of action. One of the key challenges that has been discussed in the health ML space is the lack of available crowd-sourced data for building accurate and comprehensive image classification models to build dermatology models. As a result, I wanted to explore the possibility of utilising synthetic data generation through generative AI, specifically the DALL-E image generation model. The approach involved developing a simple prompt that included placeholder inputs for the model to vary, providing the model with the goal of developing a diverse dataset, and setting the context to which the data would later be used. The approach shows immense promise with DALL-E presenting a range of accurate and representative images for acne, dermatitis, and rosacea (these were chosen at random). All data was labelled and Apple’s CoreML framework was applied using the CreateML application on MacOS. Test data was sourced from the SCIN dataset, released by Google Research Laboratory [source].

Note 1: It must be emphasised that this does not in any way seek to replace the need for consultation advice from board qualified dermatologists, but instead aims to augment the patient support process.

Design of Prompt

LLMs are often used to perform search requests by a lot of individuals, or to write code. One of the other ways it can be utilised is to use prompts with placeholders, to which the user can then specify the range of inputs to those placeholders. This is similar to designing a function in code where the output format is defined first (in this case an image), along with the inputs to the function. In this case, we need to develop a diverse dataset (e.g race, gender) with a range of severity markers (from none, through to mild, and severe). It’s also important in the case of acne to define the type, since it can vary between blackheads, whiteheads, cystic pustules, and more. The use case for this model is for a user to capture their daily progress on acne treatments, with the model then defining a marker of how severe the acne might be. This might help to build a timeline for the user of their acne progress and support dermatologists in understanding whether a patient is responding to the care prescribed.

As such, Figure 1 below shows the prompt that was used for the DALL-E image generation process.

Figure 1: DALL-E Prompt Designed for Generation of a diverse acne dataset

To showcase the outputs from this model, Figure 2 shows two examples, one on the left for moderate acne, and one on the right for very mild with a range of different acne types included within the image.

Figure 2: Two examples of the output from DALL-E for the prompt above. Left: Mild acne in a range of areas, male. Right: Acne Rosacea with scattered blackheads and whiteheads.

It’s important to note that the LLM decided on the inputs to the prompt, as the Developer, I did not define the inputs. However, a goal line was added to the prompt to encourage the LLM to diversification of the dataset to maximise image classification model performance.

Output from DALL-E

Next, once the examples were generated, images were then generated on mass, with the LLM prompted to consider further conditions such as dermatitis and rosacea as opposed to just acne. Further examples can be shown in Figure 3 below.

Figure 3: Examples from all outputs, including Acne, Rosacea, and Dermatitis. While some of the outputs often look exaggerated, as DALL-E improves, it’s likely these outputs will get increasingly accurate.

Model Performance

From the initial building of the model within CreateML, the model performance varies. With some images, we get 85% accuracy all the way up to 100%, however for more complex conditions such as Rosacea and Dermatitis, which to an image can often appear very similar (e.g. the distribution of redness, and the severity), the model seems to mistake the two.

It is important at this stage to remind the reader that the intent of this model is to provide suggestions. A range of suggestive conditions could be presented to the user of the model with the recommendation to see the Dermatologist for a definitive diagnosis. However, if the patient is already aware of their condition type, then the model can simply measure the severity to support the user in documenting their progress.

Further examples of model performance to be shared later.

Further Work

This synthetic dataset will continue to be generated due to the ability to control the conditions and severity. Model performance will be optimised, and more data added.

The following can also be done:

• Add more conditions to further diversify the dataset.
• Use a combination of real world and synthetic data to ensure real-world, validated data from dermatologists is included.
• Have dermatologists assess the synthetic data and provide their diagnosis.
• Make the dataset open-source.

Conclusion

The exploration of using Apple’s CreateML framework, in conjunction with generative AI models like DALL-E, demonstrates significant potential in developing suggestive dermatology models. By generating synthetic datasets that account for a wide range of dermatological conditions and severities, the research addresses critical data scarcity issues in the health ML space. While the current model shows promising accuracy, particularly for less complex conditions, its performance highlights the need for further refinement. The generated synthetic data should be augmented with real-world data and validated by dermatologists to enhance model reliability. Continual improvement and open-sourcing the dataset can contribute to a more comprehensive and accurate tool, ultimately aiding patients and healthcare professionals in managing dermatological conditions.

Python’s Gurus🚀

Thank you for being a part of the Python’s Gurus community!

Before you go:

• Be sure to clap x50 time and follow the writer ️👏️️
• Follow us: Newsletter
• Do you aspire to become a Guru too? Submit your best article or draft to reach our audience.