Game, Set, Match: Revolutionizing Tennis Court Availability with an Innovative Human Presence Detection System
Project: Human Presence Detection System for Autonomous Occupancy Detection of Tennis Courts
--
Problem Statement
According to the latest statistics from the United States Tennis Association, there are an estimated 270,000 courts across the country to cater to the 23.6 million tennis players. As an avid tennis player myself, I have experienced the challenge of locating available courts, as information on court availability is often limited or inaccurate. This results in wasted time spent driving around to different locations or waiting for a court to become free. In light of these limitations, I began to consider the use of technology as a solution to this problem. The development of a mechanism that can accurately provide information on the status of various courts, such as whether they are occupied or vacant, as well as an estimation of when players may finish their games, would be of great value to tennis enthusiasts. By leveraging technology to address these challenges, the game of tennis can become more accessible and enjoyable for players across the country.
Several solution options were considered to solve this problem.
1. To manage court usage, one solution is to create a reservation system. However, this approach may have some challenges. Some players may reserve a court but need to show up or stay for a shorter time than they indicated. Additionally, some players may prefer the flexibility of playing whenever they want without having to reserve in advance.
2. A different solution is to create a check-in system at the courts and ask players to specify their checkout time. Then I would use that information to display to users when players have checked in and estimated departure time. However, most users may forget to check in and it is difficult to change or encourage user behavior. Thus players may still end up going to the courts and find it occupied.
3. A third solution is to implement a human occupancy detection mechanism that can be installed for every court to detect the real-time status of courts. There are various methods of detecting occupancy such as IoT devices consisting of various sensors, cameras, AI/ML, etc. This option would be most reliable as it solely depends on the accuracy of the device itself, not on human cooperation.
Thus, I decided to follow through with the research using this approach — human presence detection with IoT systems.
Research Question
What is the most effective and economical approach, utilizing machine learning algorithms or electrical engineering techniques, to accurately predict human presence and occupancy on a tennis court?
Methods
I. Methods for Human Presence Detection. Various methods are available for detecting human presence, including video, audio, or sensor technology. In this study, I tested several such methods, namely video with machine learning human detection (computer vision), microwave sensing with Arduino, passive infrared sensing with Arduino, and Bluetooth detection with ESP32. I chose these methods because they are cost-efficient and are known to accurately predict human presence. While other options like machine learning with the NVIDIA Jetson Nano or the X4M300 UWB sensor might provide better performance, their higher cost makes them impractical for scaling to multiple courts.
II. Method #1 — Passive Infrared Sensing with Arduino. A passive infrared (PIR) sensor is an electronic device that detects the infrared radiation from objects within its viewing area. They operate by detecting changes in heat in the environment resulting in a binary output. Typically utilized in motion detection applications such as automated lighting controls and security systems, PIR sensors can be connected to an Arduino microcontroller and tuned with delay and sensitivity parameters. This integration is facilitated by the straightforward digital input/output pins required for interfacing with Arduino, rendering PIR sensors an inexpensive, dependable, and easily configurable option for motion detection.
For method testing, I employed the HiLetgo HC-SR501 PIR Infrared Sensor. Specifications — operating voltage range: DC 4.5–20V, angle: <100° cone angle lens, maximum range: 10m. See Fig. 1 for the circuit diagram.
III. Method #2 — Microwave Sensing with Arduino. Microwave sensors, also known as radar or Doppler sensors, use radio waves to detect the presence and motion of objects. Specifically, they operate under the Doppler effect, where the antenna on the sensor emits radio waves at a certain frequency (3.2GHz for the RCWL-0516), and the strength of the signal that returns depends on the size of the object that reflects it. Such sensors are widely used in motion detection, proximity sensing, and level sensing applications. When a microwave sensor is integrated with an Arduino microcontroller, the sensor can return a binary output true or false.
III. Method #3 — BLE Sensing with ESP32. BLE (Bluetooth Low Energy) device sensing is a cutting-edge wireless technology for communicating with Bluetooth devices. This technology involves the utilization of small, low-power devices like smartphones, smartwatches, or other wearable devices equipped with a BLE chip. These devices establish communication with BLE sensors placed around a room or building, which detect their presence and transmit this information to a central system. In theory, if a Bluetooth device were to be detected at a sports court, this could suggest that a person is present at the court, as most players will carry a phone, headphones, or other devices.
III. Method #4 — Machine Learning with Camera and Raspberry Pi. Live video classification, specifically utilizing machine learning and a Raspberry Pi (computer vision), is a technique employed to analyze live footage. This technology employs a Raspberry Pi, a small and low-power computer, to process video data in real-time. The Raspberry Pi, equipped with a camera module, would run a machine learning algorithm to analyze the video and classify the objects in it. The system can be trained to recognize human characteristics, such as body shape, movement, and other features, thereby enabling it to determine the presence of people in a particular area and transmit the information accordingly. I explored YOLOv3 as the ML model, which uses a single convolutional network that predicts bounding boxes and the resulting probabilities. In comparison to the Jetson Nano, which may be more efficient and powerful, the Raspberry Pi and Pi Camera combination offers a cost-effective solution that can be easily scaled.
Results
I. Method #1 Performance. To evaluate the performance of the HiLetgo HC-SR501 PIR Infrared Sensor (operating voltage range: DC 4.5–20V, angle: <100° cone angle lens, maximum range: 10m), I coupled the sensor with an Arduino Uno microcontroller in accordance with the circuit diagram illustrated in Figure 1. The experimental design consisted of several trials with the experimenter positioned at varying angles, distances, and postures in front of the sensor. Our results indicated that the sensor yielded higher 78% accuracy in detecting significant movements such as walking, running, or shuffling, but exhibited a modest 68% accuracy rate in detecting minor actions such as waving a hand, moving slowly, or standing still, as evidenced in Table 1. False-negative outcomes were common, meaning that the sensor did not detect movement despite human presence.
II. Method #2 Performance. Our experimental design employed the RCWL-0516 Microwave Radar Sensor (operating voltage: 4–28V, detection distance: 5–7m) coupled with an Arduino Uno microcontroller, as per the circuit diagram depicted in Figure 3. Multiple trials were conducted with the experimenter positioned at varying angles, distances, and postures in front of the sensor. Our testing revealed 82% accuracy, as detailed in Table 2.
III. Method #3 Performance. For testing purposes, the ESP32 microcontroller was utilized with its built-in BLE chip. The device was programmed to consider a Bluetooth device present if it received a signal of -60dB RSSI. To test its efficacy, an iPhone XR was dropped in various locations within the range of the device, resulting in 100% accuracy, as shown in Table 3.
IV. Method #4 Performance. To evaluate the performance of the computer vision method, I utilized the Raspberry Pi 4 Model B with a Picamera v2. To run the YOLOv3 machine learning model, the OS of the Pi was downgraded to Buster. The model is known to demonstrate an accuracy of 98.87%, with a recall rate of up to 98%. These findings were corroborated during my personal testing (100% accuracy), as illustrated in Table 4.
Discussion
I. Method #1 Evaluation. Overall, the PIR sensor performs quite accurately in sensing larger human motions, as evidenced by its 78% accuracy. However, its 68% accuracy in minor movements and detecting presence present problems for the use case of functioning self-sufficiently on a tennis court. Primarily if a person is standing still or sitting down, a common occurrence at a tennis court, the sensor will create false negative values. Additionally, being an infrared sensor, it can also detect various other animals including squirrels or birds, causing false positives to occur. Buying an upgraded or more expensive version comes with the same drawbacks as well, as ultimately passive infrared only detects changes in motion. Thus, while it is a powerful sensor, it is not the best option for the use case of detecting tennis court occupancy.
II. Method #2 Evaluation. The high testing accuracy of the microwave sensors proves that they are a valid option for human detection microwave sensors, also known as radar or Doppler sensors, use radio waves to detect the presence and motion of objects. These sensors have significant advantages over PIR sensors, as they can detect human presence and motion more accurately due to the Doppler effect and their continuous sensing capability. Microwave sensors are capable of detecting objects at a greater distance and can operate in harsh environments, making them ideal for industrial and commercial applications. However, like the PIR sensor, microwave sensors can detect other objects such as birds and squirrels, which can cause false positives.
Despite their advantages over PIR sensors, microwave sensors are still not the best option for detecting human presence on a tennis court. One issue is that their continuous sensing capability can lead to higher power consumption compared to other sensors, which may not be feasible for self-sufficient installations. Additionally, microwave sensors are more expensive than PIR sensors, and their installation and maintenance require specific technical skills. Another concern is that microwave sensors are not effective in detecting small objects, such as a ball or a tennis racket, which can lead to inaccurate occupancy detection. As such, while microwave sensors offer superior sensing capabilities compared to PIR sensors, other sensor options may be more suitable for specific use cases such as tennis court occupancy detection.
III. Method #3 Evaluation. BLE sensing is a highly accurate technology for detecting human presence, however, its effectiveness is limited by a fundamental assumption it makes: the assumption that all individuals present in the area being monitored will be carrying a Bluetooth-enabled device and have it turned on. This assumption is based on the premise that such devices are ubiquitous and that individuals will always have them on their person. However, in reality, this assumption is not always valid as human behavior can be highly variable and unpredictable. For instance, individuals may forget or choose not to carry their Bluetooth-enabled devices with them, or they may switch off their devices to save battery life, thereby rendering the BLE sensing technology ineffective. Consequently, while BLE sensing can be an effective technology for certain use cases, it may not be the optimal choice for situations where the presence of humans needs to be accurately monitored, such as on a tennis court.
IV. Method #4 Evaluation. The use of computer vision with YOLOv3 has proven to be a highly accurate approach for detecting human presence on a tennis court. However, privacy concerns arise with the use of cameras, as players may not wish to be video recorded/stored while playing. One solution to this problem is running the ML model on the edge, meaning the Raspberry Pi performs live video classification without storing any data or images long-term in the cloud but instead discarding them after classification. This method eliminates the risk of violating players’ privacy while providing a high level of accuracy in detecting human presence. As mentioned, according to prior research studies, the YOLOv3 model achieved an accuracy of 99.1% and a recall rate of 98.8% for human detection. Furthermore, this method is not susceptible to false positives due to the high precision of its human detection. As a result, using a Raspberry Pi with a camera, paired with computer vision, is currently the most reliable and accurate method for detecting human presence within a tennis court.
Final Design Architecture
Now that it has been proven that computer vision with the Raspberry Pi is the best method of detecting human presence, the next step is developing the prototype for it to run autonomously at a tennis court, and setting up communication of occupancy through the cloud such that it can be displayed to users. Below is the design architecture that I crafted:
Hope you enjoyed reading about my project! I had a lot of fun working on it :)
