Building your own interactive touchscreen table with object recognition

Andreas Schallwig
Jun 11, 2019 · 7 min read

Recently there’s been a kind of touchscreen table application getting more and more popular at events and showrooms. People interact with an application by putting objects directly onto the screen. The touchscreen will then immediately recognize the objects and react by showing information related to the objects.

Example of an interactive touchscreen table application with object recognition

The objects can also be moved around or act as dials to interact with the touchscreen user interface. This allows for some interesting and creative UI concepts which integrate real-world objects as part of the user interaction.

It’s pretty cool stuff — So let’s have a look at how this is actually done (and build one ourselves!)

Touchscreen table object recognition methods — A comparison

There are multiple ways to achieve this depending on your budget and requirements:

Cameras combined with real-time image recognition software will give you great flexibility in your choice of objects and eliminate the need for mounting the objects on a base disc but it is also (much) more expensive to setup and maintain than touch-only based solutions. Use this solution if you need large numbers of objects to be distinguished and you have the 💰💰💰 to spare.

Capacitive multi-touch screen panel with base discs are a great (and budget-friendly) solution if you are looking for only having a limited number of objects on the table at the same time and do not require many different objects to be distinguished. In this case you could go for a touch-only solution as shown in the video below:

Touch-only solution with objects mounted on a base disc

The touch-only solution you can build by yourself completely from off-the shelf parts combined with a bit of math and Maker-fu.

There are some limitations though:

  • Your objects need to be mounted on a base disc (see the video above)

Most touchscreen table UI concepts will work perfectly fine within the limitations above, still you need to make sure that none of the above is a showstopper for you. Simply because there aren’t really any workarounds available.

“A bit of math and Maker-fu”

As mentioned above your objects will need to be mounted on a base disc in order for the screen to recognize and distinguish the object. How does this actually work? The answer is not 42 but “Triangles”.

The base disc is merely emulating three finger presses on your touch screen making up a triangle. We won’t use any triangle but we will use isosceles triangles with distinguishable vertex angles. These angles can then be mapped to different objects. So you can later say “A 20 degree vertex angle maps to object A while a 36 degree vertex angle maps to object B”.

An isosceles triangle

This approach also explains the limitations described above. Since most multi-touch screens only support up to 10 fingers at the same time we can only use three objects simultaneously as they will already take up 3x3 of the available touch points. Also as the theoretically maximum vertex angle must be less than 180 degrees, there are only about 10–12 distinguishable angles you can use.

Note: You might be able to squeeze a bit more angles out of this setup but from our experience results are increasingly unstable if you go beyond 12 different angles. If you really need more than 12 objects try adding the triangle height as an additional dimension for your object recognition but YMMV.

In order to make this work we need to create a small piece of software with the following core functions:

  • Accept a set of coordinates (our touch points)

To make your life easier I have already implemented a JavaScript based library which you can simply drop into your own project and get started without dealing with any of the math. You can download the library from my GitHub repository. The repository also includes an HTML5 demo application which will let you test your discs.

Let’s start making!

Now with the math stuff out of the way we can directly jump to the fun part — Building your own object-recognition touch table!

What you will need:

Hardware:

  • A capacitive multi-touch capable LED screen (42’’ or above recommended)

For the DIY discs:

Casting the discs

  • Use a measure cup to mix both resin components (epoxy + hardener) according to the required proportions (usually 2:1 or 3:1 — check the instructions)
Resin casting process

Once the resin has set take out your discs and ensure the lower surface has come out flat. If it’s flat you can move on to the next part.

Note: In my pictures you can see both “solid” and “donut” style discs. Both will work fine but after a couple of projects I prefer the solid ones. The donut shaped discs (while lighter) tend to bend or shatter if they are dropped on the floor.

Adding the conductive spots

The next step is to make your disc electrically conductive so it can actually emulate three finger presses. We will use the copper tape for this:

  • Wrap one strip of copper tape all around the outside of the disc, but DO NOT CLOSE the “loop”. Means you should leave a little gap of about ½ inch between both ends (see picture).
Adding the conductive spots to your disc. Example pictures show 30 and 54 degree angles

Note: There are other ways to create the conductive touch points. If you require a bit more precision you could for example use conductive small foam pads on top of the copper tape.

I suggest using angles of 18, 36, 54, 72, 90, 108 and 126 degrees for your first test. Once you got the hang of it you can reduce the distance between angles to 12 degrees. Just don’t use 60 degrees as this will create an equilateral triangle and you can’t track its rotation.

Testing your discs

Once you have finished your discs you can test them using the example HTML5 page from my GitHub repository.

Happy making and I’d love to see your creations!

Our DIY object markers in action \o/

The Startup

Get smarter at building your thing. Join The Startup’s +793K followers.

Sign up for Top 10 Stories

By The Startup

Get smarter at building your thing. Subscribe to receive The Startup's top 10 most read stories — delivered straight into your inbox, once a week. Take a look.

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

Check your inbox
Medium sent you an email at to complete your subscription.

Andreas Schallwig

Written by

Co-founder of multiple IT start-ups, passionate about blinking stuff, computers and cooking. Currently working as Technical Director at mediaman in Shanghai.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +793K followers.

Andreas Schallwig

Written by

Co-founder of multiple IT start-ups, passionate about blinking stuff, computers and cooking. Currently working as Technical Director at mediaman in Shanghai.

The Startup

Get smarter at building your thing. Follow to join The Startup’s +8 million monthly readers & +793K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store