Interactive gaming with AI and scratch

What is Artificial Intelligence?

Artificial Intelligence is a process of programming computers to make decisions for themselves. This means creating algorithms, or sets of rules, to sort, study, and draw predictions from data. It also involves making decisions based on data, learning from new data, and improving over time.

Artificial Intelligence is included in the Computer science curriculum in many universities, schools, and after-school clubs. It is a process of programming computers to make decisions for themselves. This means creating algorithms, or sets of rules, to sort, study, and draw predictions from data. It also involves making decisions based on data, learning from new data, and improving over time. That’s why it’s important to learn AI. More detailed information on why kids should learn AI can be accessed here.

What if we told you that you could learn the basics of AI with some interesting projects. In this tutorial, we’ll show you how to use Scratch and machine learning together to create a simple game. You’ll learn how to train a Scratch project to recognize images and make decisions based on what it sees. By the end of this tutorial, you’ll have created an AI-powered Scratch game that you can enjoy with your friends.

What is Machine learning?

Machine learning is a subset of AI that deals with the creation of algorithms that can learn from and make predictions on data. Machine learning focuses on the ability of computers to improve their performance on a specific task without being explicitly programmed to do so.

Machine learning is a subset of AI that deals with the creation of algorithms that can learn from and make predictions on data. In machine learning, we give computers data (this is called training data) and then let the computer figure out the rules by itself. The goal is to have the computer learn enough so that it can make good predictions on new data (this is called test data).

You can think of machine learning as teaching a child how to recognize animals. First, you would show the child some pictures of animals and tell them what they are. Then, you would give the child some new pictures of animals and see if they can correctly identify them.

As the child sees more and more pictures of animals, they will start to learn the characteristics that make an animal an animal. After seeing enough examples, the child should be able to correctly identify a new picture of an animal most of the time.

This is how machine learning works. We give the computer training data, which is a set of data that we already know the answers to. The computer then figures out the rules by itself and applies those rules to new data (test data) to make predictions.

Machine learning is often used for tasks like image recognition, facial recognition, sentiment analysis, and fraud detection.

Let’s get started with Machine learning first

For creating a machine learning model we will be using a web-based tool Google teachable machine. Teachable Machine makes it easy to train your own machine learning models, regardless of programming experience.

With Teachable Machine, you can train an image classification model to recognize different objects in pictures, a sound classification model to distinguish between different sounds, or even a body pose estimation model to track the movement of people in images.

You can then use these trained models in your own projects, such as a Scratch project that uses image classification to control sprites or a website that uses sound classification to play different sounds depending on what the user is saying.

Best of all, Teachable Machine is completely free and easy to use! So let’s get started. First, go to the teachable machine and click on get started.

You will be asked to select a project type. For this tutorial, we will be using the Pose recognition project type. Click on the Pose project.

On the next screen, you will be asked to name different categories or classes of data. In this project, we will be creating three different classes such as tilt left, tilt right, and neutral.

After creating three classes we need to provide samples or data to respective classes. To provide data to the machine learning model we can open our webcam to click images of our left (head tilt left), right (head tilt right), and neutral pose.

After providing the dataset we need to train our model. When the training is over we can preview our model. This is like we are testing the model. So, we can tilt left, tilt right and give neutral poses (this is test data) in front of the webcam. The machine learning model will provide the result according to the test data.

Export model

After testing it’s time to export our model and use it in Scratch. Click on the Export model link. On the next screen, we need to click on Upload, my model. When the uploading is complete we will get a link that we can copy (Click on the copy button as shown in the image).

Now Turn Off your webcam on the Teachable machine using the toggle button shown in the image below.

Use Model in Scratch AI interface

Now, we need to open another browser-based application i.e. PRG AI blocks. The link to the Scratch similar site which contains AI blocks is written below

https://mitmedialab.github.io/prg-extension-boilerplate/create/

The website will take you to a Scratch-like environment with AI extensions. The next step is to add the extension. For adding an extension we need to click on the button located at bottom of the page.

When you click on extension buttons you will go to a page with different extensions. Now press on the teachable machine extension to get the teachable machine blocks.

The next step is to take two sprites. We have taken a wizard girl and a star sprite in this project.

We will code the Wizard girl first. The wizard girl needs to move left when we are tilting toward the left (in front of the Webcam), move right when we are tilting towards the right and when a neutral pose is there should not be any change in direction of the Wizard girl sprite.

We need to activate the machine learning model (created in Teachable machine) in the Scratch AI interface. For this, we need to paste the link in the Teachable machine block as shown in the picture below

We will use the event handler “when green flag clicked “ with the use model block to activate the machine learning model. As the model is activated you will see a green tick (as shown in the picture). Also, the classes will appear in the teachable machine blocks (as shown in the picture).

The complete code for the wizard girl is shown below for your reference. We will create two variables (i) direction which will help in the movement of the sprite and (ii)Score which gets incremented when the wizard girl catches the star.

Also, we need to code the stars so that they fall from the top of the screen and appear in random places.

The clone for the stars is created in the backdrop. The code for the same is shown in the picture below.

It’s time to enjoy the game. When you are ready, click on the flag button to start playing. You can see the live classification of your pose in the stage monitor if the model prediction block is checked ( in the Teachable machine extension).

As you tilt your head, the direction of the Wizard girl changes. The wizard girl moves left when you tilt your head towards the left and the wizard girl moves right when you tilt your head toward the right. The aim of the game is to catch as many stars as possible. Good luck catching the stars!

Conclusion

Making machine learning projects with Scratch is so fun and easy. In this project, we saw how to use Scratch and AI together to make an interactive game. We trained a machine learning model using Teachable Machine and used it in Scratch. Our students at SkoolofCode are having fun learning AI concepts using Scratch extensions. This is just one example of the many possibilities that Scratch and AI offer. So try out different things and see what you can create!