Getting started with Computer Vision AI / ML — Tutorial Step 7 of 7: Create Actions that are triggered based on what the camera sees

In this tutorial, we learn how to create a system to detect if a sink contains any dishes that need washing up. This is step 7of 7. Start with Step 1 here, or view the full tutorial on Gravio.com.

Steps to get into Machine Learning and Computer Vision

1. Collecting images of as many different situations as possible (Details)
2. Setting up your Google Cloud Vision account (Details)
3. Uploading and labeling those images via the Google Vision AI website, training the Model (Details)
4. Download the model from Google’s platform (Details)
5. Set up Gravio Coordinator and connect HubKit to it (Details)
6. Deploy the model to Gravio Coordinator and subsequently to HubKit (Details)
7. Create Actions that are being triggered based on what the camera sees 👈 YOU ARE HERE

Step 7 — Create Actions that are being triggered based on what the camera sees

Do you remember how we created the Area and Layer in Step 1? Go back to that layer and delete it:

Then re-add that newly available layer along with the camera:

Add the camera again and enable it:

That’s it, if you click on “Data” and enable the “Live” switch, you should see now data coming through depending on what the camera sees:

If there are items detected, you will get labels and coordinates. If you there are none detected, you will get "detections":[]

Now, let’s trigger an Action if something is detected. First let’s create a random Action. In this case here, we just play a sound:

We trigger the Action that plays an alert sound, if the length of the items are greater than 0.

That’s it, you have now connected your Action with your Trigger, your Trigger with your Camera, and your Camera with your AI model. Happy object detecting!

Summary

Creating your own models has become very easy with Gravio. The biggest part of the work is setting up, collecting images and labeling them for training.

But being able to create your own image recognition models is very powerful, especially if you don’t need specialised and expensive gear.

Please note that the quality of your AI detection system will heavily depend on the quality of your training images and your labelling. Machine Learning is never finished. Keep improving your models. And please keep privacy in mind when working with cameras filming people, even if you’re not storing the images in production.

Join our Slack if you have questions or feedback regarding this tutorial.

Go to the full A-Z tutorial on Gravio.com