Make money using NFT + AI | Feature Visualization of model

Published in

Code Sprout

6 min readMay 8, 2021

Introduction

If you are new to NFT, please read these articles to understand it. This article will discuss how to create a unique type of art that can be minted. I will go over the basic to advanced version to generate art. For this article, I will use feature visualization and show how to create images out of it.

Setup

To create images, you would need a decent PC setup with GPU. If you do not have one, you can use any cloud instances or use the Google Colab environment. Throughout the article, I will be using PyTorch for all generations. Please feel free to use Tensorflow as well.

Feature Visualization

If you have worked on AI for some time, you would have seen or worked on style transfer. It is one of the well-known methods to convert an actual image to a modified art image.

Similar to the style transfer, feature visualization can be used to generate modern art. When you pass an image via different convolution layers like a pre-trained model such as inception v1, the internal layers will capture the representation of the image. Some layers will find out the textures and patterns of the image, and some other layers will find the shape of the image. Likewise, a pre-trained neural network will have many different layers, and each layer will hold some pattern of the image.

Different **optimization objectives** show what different parts of a network

In the above image, you can see that the first picture is a visualization of a single neuron converted into an image. Similarly, subsequent images show the visualization of different layers in a neural network.

The above illustration shows that you can provide a list of images and visualize the features in a pre-trained model. It shows how each layer dreams about the image. Some images may have the pattern of output images, but some might not have. You can check the detailed information from this link.

The interesting fact here is you do not need an input image to visualize the internal feature visualization of a pre-trained model. In a pre-trained model, the weights are already trained. Even if you provide a noise as an input, the weights will create some representation of noise.

Don’t worry if you do not understand any of the above. The simple information is that a pre-trained image-based model can visualize an image in various forms and provide us the final result of the neural network. However, it might not make sense to us by gathering all the different versions of images will give a final result.

OpenAI Microscope

OpenAI Microscope is a collection of visualizations of every significant layer and neuron of several common “model organisms,” which are often studied in interpretability. The microscope makes it easier to analyze the features that form inside these neural networks. It will help the research community to move towards understanding these complicated systems. You can check out at this link.

Visualize different models:

OpenAI microscope has various models. As Inception v1 is a popular model, let’s visualize it. Here the input is a plate of pasta image. You can see how each feature visualizes and understands the image.

Different layers visualization of pasta plate

When you go towards the end, the pasta image is mostly not recognizable from the human eye. There are some traces of it. However, the most of the image is of a different color. You can check it from this link.

What we are interested in on the feature representation of the layers inside it. These can be considered as art! It represents something in a different form. Now let’s see how to get it and can we sell it.

Create dreams or art

Here we are going to use PyTorch as our core library and use the torch-dreams package.

pip install torch-dreamsAlso, import other required packages, and we have to import a pre-trained vision-based model to visualize the internal features. In this case, we will import the inception v3 model.

Voila! You became a digital artist! You will see an image like the below one. This one is generated from the noise as an input image. You can alter the layer in line 7, and if you output the model layers, you can see an extensive list. You can go deeper to any levels.

If you go one level deeper, you will get a different representation of the feature.

If you are looking for some more options and wanted further to customize the height and width with additional parameters, the below code will serve your needs.

Recently minted item at Opensea https://bit.ly/3tvsQtY

Check the above creation! We are an artist now. I have also listed it in the opensea. We can also write your custom function and pass the custom function to torch-dreams function dreamy_boi. Layers can be tuned and changed accordingly to get more designs. Here dpi represents the resolution of the image. Remember to save the image before plt.show() function.

Please feel free to change the height and width according to your artist within you!

With your own image

Here the question is, will own this noise according to NFT? It should be something unique. Possibly not, however in this section, we will walk through using custom images.

In the above section, animals.jpg is a custom image that we can use to get the internal representation. Here we hold the absolute rights(except model) of the image to modify it to any different version of it.

You can also modify the model you use and a combination of different layers to form an image. Below is another art that I created with a combination of different layers.

There are numerous options if you wanted to create modern images according to your need. You can also create caricatures of a person or celebrity.

Conclusion

Develop art using neural networks. We need to experiment with it with different layers like a chemical reaction with the right proportion.
There are so many models with different layers. We need to pick the correct one to visualize.
One issue is the size or resolution is not so great. I will address it in my next article.
This type of modern art can be developed and sold on multiple NFT platforms.

Disclaimer: I am showing the approach and it is purely for informational purposes. It is not guaranteed that you will make money. You may also not be able to produce unique arts via AI. It really depends on the input dataset and your skills

Series of related articles to read

Reference links

Get Code

Please subscribe to my newsletter to get the free working code for my articles and other updates.