How to turn old Kinect into a compact USB powered RGBD sensor

Kinect 360, not so popular among gamers, but is a popular and cheap sensor among robotics hobbyists. It provides environment depth information — like a camera but 3-dimensional. Kinect 360 is based on a PrimeSense chip (same as in Asus Xtion sensor was) and can provide point cloud with RGB information.

Open libraries for Kinect 360 and ROS support

One of the popular and open source libraries is libfreenect by OpenKinect project. There is a ROS package which wraps libfreenect into a ROS node. How to build this node for ROS Melodic I will show later in this post.

Looking into Kinect 360

Kinect 360 has proprietary connector. But in the fact it is a USB 2.0 connector with additional 12V power input. So it is easy to replace proprietary plug with USB type A connector and 12V AC input:

Why Kinect 360 needs 12V? Besides the depth sensor, Kinect 360 has microphone array, accelerometer (well, this chip doesn’t need much power), tilt servo-motor, descent cooling fan, USB hub to tie all subsystems in one piece… All this stuff needs power, and in Microsoft decided to add one more power bus.

The electronic part consists of three boards:

• Main board, which unites two other and contains Kinect processor, 64MB of RAM, microphone pre-amplifier, USB Hub. External USB cable and 12V power wire are connected to this board.
• A small board with audio controller and accelerometer
• And the board, I’m interested in, is that one with PrimeSense PS1080 chip. Optical sensors and infrared emitter are connected to this board also.

There is a 14 pin port on the optical sensors board for connection to the Main board. Asus Xtion depth sensor worked on the same chip and had no any additional peripherals to connect to USB, so I assumed that this 14 pin port should have a USB data pins. Information from the Internet and logical analyzer confirmed my assumptions. Long story short, below is pinout for this port:

OK, but what’s the idea?

If we have a USB port on the front board, why not to have direct connection to it, bypassing Kinect’s USB hub. In this case audio, tilt servo and accelerometer subsystems can be removed. This will significantly reduce Kinect’s body size.

Adding a little bit of electronics

The optical sensors board needs three different voltage inputs: 1.8V, 3.3V, 5V.

5V can be taken directly from USB bus, for the rest we need a step-down converters. First what I’ve found at home was LM2596 buck converters and they worked just fine.

First launch

As a test computer I will use Raspberry Pi 3B+ with Ubuntu Linux 18.04 and ROS Melodic installed. Note, that Kinect sensor draws about 500–700 mA. This current can be critical for some USB 2.0 ports! So do this experiment on your own risk!

Let’s check log file after connecting sensor to USB port by typing dmesg:

Records in syslog confirm that Kinnect 360 sensor is connected and recognized by system. Next step is to install Kinect drivers for ROS.

Install freenect_stack in ROS Melodic

For testing Kinect “Lite” sensor I will use libfreenect library and freenect_stack ROS package. Today ROS Melodic does not have freenect_stack package binaries in repository, so we need to compile and install it manually. There are two steps: build freenect library and then build freenect_stack package.

Folder will contain catkin workspace and directory libs for additional libraries' sources. For freenect library successful compilation glut and libusb dev packages should be installed.

mkdir -p ~/kinect_ws/src
mkdir -p ~/kinect_ws/libs
cd ~/kinect_ws/libs
git clone https://github.com/OpenKinect/libfreenect.git
cd libfreenect
sudo apt install freeglut3-dev libusb-1.0–0-dev
mkdir build
cd build
cmake -L ..
make
sudo make install

After successful library installation everything is ready for ROS package installation. Go back to the workspace src directory, clone ROS package sources and build. Before building I recommend to check if all the dependencies are installed by rosdep tool.

cd ../../../src
git clone https://github.com/ros-drivers/freenect_stack.git
cd ..
rosdep install — from-paths src — ignore-src -r -y
catkin_make

That’s it, now we are ready to run Kinect with ROS. Don’t forget to declare new workspace in the environment first:

source ~/kinect_ws/dev/setup.bash
roslaunch freenect_launch freenect.launch

In rviz we can check the result:

What can be improved

Proof of concept works and of course I already have ideas what can be improved:

• First of all, create a compact sensor body. As an option to add mounting holes compatible with Realsense sensors or Go-Pro mount;
• Use smaller buck converters, or create a custom board, which will include 3.3V and 1.8V converters and a USB port;
• Instead of soldered USB cable I’m going to use a micro USB port;
• And maybe it makes sense to return the cooling fan. I need to check the sensor’s temperature during operation.
• Additional 5V power input. According to the information found on the Internet, peak current draw can be up to 950mA, which is above what can be ‘legally’ drawn from a single USB connector. So it is reasonable to make a dedicated 5V input port.

Links

Hardware info: https://openkinect.org/wiki/Hardware_info