KatWalk C2: part 2: peeking, eavesdropping, sniffing and learning how to communicate with unknown hardware using unknown language
In the previous article, I’ve explained what the VR Treadmill is plus how to get the data from it and how to inject the communication
code right into 3rd party binary (game).
Unfortunately, getting the data this way requires constantly running a background application (C#), which means it is worth understanding what
the application is doing!
Let’s grok it up and get rid of the middleware!.. By rewriting it in Kotlin. Why Kotlin? Because I never wrote on Kotlin before.
What’s the `extraData`
But first things first: the SDK also gives us `extraData` array, what’s that?
So, that’s how the structures are defined in SDK:
struct DeviceData
{
bool btnPressed;
bool isBatteryCharging;
float batteryLevel;
char firmwareVersion;
};
struct TreadMillData
{
char deviceName[64];
bool connected;
double lastUpdateTimePoint;
Quaternion bodyRotationRaw;
Vector3 moveSpeed;
};
struct KATTreadMillMemoryData
{
TreadMillData treadMillData;
DeviceData deviceDatas[3];
char extraData[128];
};We have normalized Quart for looking direction (the “Calibration” button that the player can hit any time on the backplate of the treadmill, from the gateway interface, or just simultaneously pressing both triggers on controllers already applied here); movement vector and information about connection status with last update timestamp. Additionally, there is information about sensors — their charge level, firmware version and status of button press. Nothing of the real interest.
But there is also `extraData` array, having some data! So, let’s find out its mystery. Direct search inside SDK files doesn’t give anything, so let’s run dotPeek again (with all the binaries still open since last time) and search (Ctrl+F) using “extraData”. Hm. Nothing?!..
What about `Navigate->Search everywhere` (Ctrl+T)..? “ZipExtraData” inside of “SharpZipLib”. Nope, not. Althoooough… looking by just “extra” we see:
extraInfo in KATSDKInterfaceHelper
extraInfoLoco in KATSDKInterfaceHelper
extraInfoMini in KATSDKInterfaceHelper
...Great! So that’s just copy-paste issues. Looking closer, indeed, it’s the things we want:
[StructLayout(LayoutKind.Sequential)]
public struct extraInfo
{
[MarshalAs(UnmanagedType.U1)]
public bool isLeftGround;
[MarshalAs(UnmanagedType.U1)]
public bool isRightGround;
[MarshalAs(UnmanagedType.U1)]
public bool isLeftStatic;
[MarshalAs(UnmanagedType.U1)]
public bool isRightStatic;
[MarshalAs(UnmanagedType.U4)]
public int motionType;
public KATSDKInterfaceHelper.Vector3 skatingSpeed;
public KATSDKInterfaceHelper.Vector3 lFootSpeed;
public KATSDKInterfaceHelper.Vector3 rFootSpeed;
}
...
public static KATSDKInterfaceHelper.extraInfo GetExtraInfoC2(
KATSDKInterfaceHelper.TreadMillData data)
{
GCHandle gcHandle = GCHandle.Alloc((object) data.extraData, GCHandleType.Pinned);
try
{
return (KATSDKInterfaceHelper.extraInfo) Marshal.PtrToStructure(gcHandle.AddrOfPinnedObject(), typeof (KATSDKInterfaceHelper.extraInfo));
}
finally
{
gcHandle.Free();
}
}That means the extraData is just a placeholder for some union of structs of multiple types: extraInfo / extraInfoMini / extraInfoLoco, depending on the actual locomotion solution by KAT used. It is a little strange then why C2 and C use the same struct since C is based on IMU and C2 is based on optical sensors.
Anyway, let’s search the usage of the structs…
// ...
if (Home_Form_Walk_C2_Main.objextraInfo.isLeftGround && ((double) Home_Form_Walk_C2_Main.objextraInfo.lFootSpeed.x != 0.0 || (double) Home_Form_Walk_C2_Main.objextraInfo.lFootSpeed.z != 0.0))
{
float num8 = Home_Form_Walk_C2_Main.objextraInfo.lFootSpeed.x * 3f;
float num9 = Home_Form_Walk_C2_Main.objextraInfo.lFootSpeed.z * 3f;
Home_Form_Walk_C2_Main.LF_Foot.X += num8;
Home_Form_Walk_C2_Main.LF_Foot.Y -= num9;
}
else if (!Home_Form_Walk_C2_Main.objextraInfo.isLeftGround || (double) Home_Form_Walk_C2_Main.objextraInfo.lFootSpeed.x == 0.0 && (double) Home_Form_Walk_C2_Main.objextraInfo.lFootSpeed.z == 0.0 && (double) Home_Form_Walk_C2_Main.objextraInfo.lFootSpeed.y == 0.0)
{
Home_Form_Walk_C2_Main.LF_Foot.X = 40f;
Home_Form_Walk_C2_Main.LF_Foot.Y = 40f;
}
// ...Got it. C2 data was just stored inside the same structures: movement with X/Z plane as deltas from packet to packet. We don’t know which units are used, although. And we still don’t know where the data comes from.
Since the gateway shows the same data that `KATNativeSDK.dll` gives us, let’s see what exactly is this data. Load the library into IDA, and switch to the `GetWalkStatus`…
strcpy((char *)Buf1, "KAT_SHARED_MEM_");
// ...And so on. So we have a separate process/thread, that reads the data, processes it (obviously we need to turn the movement of two independent feet into one single smoothened body movement speed) and places it into shared memory. SDK/clients/games just read the data whenever they want (most probably — every rendering frame) from the shared memory latest snapshot.
Okay, with this knowledge we can get rid of KATNativeSDK.dll and read data directly… Well, from the gateway, using the same way. No.
How to get data directly
Time to look inside of the things. Let’s take good ol’ USBTreeView (i have it lying around in `C:\Games\`, yeah) and look at the device connected. So, we can see two KAT devices.
One is `KATVR walk c2 position — HID`, and another is `KATVR walk c2 receiver — HID`. Using the method of applying controllable force (i mean, disconnecting and reconnecting the cable back) we can discover that the `KATVR walk c2 receiver` is the platform. What’s the other one? Yeah, as I have the C2+ model, I also have a tiny USB dongle that is supposed to give the signal whenever a seat on the platform is used.
So, let’s look closer at the c2 receiver.
========================== Summary =========================
Vendor ID : 0xC4F4 (Unknown Vendor)
Product ID : 0x2F37
USB Version : 1.0
Port maximum Speed : High-Speed
Device maximum Speed : Full-Speed
Device Connection Speed : Full-Speed
Self powered : no
Demanded Current : 160 mA
Used Endpoints : 3
======================== USB Device ========================
+++++++++++++++++ Device Information ++++++++++++++++++
...
Child Device 1 : HID-compliant vendor-defined device
...
---------------- Connection Information ---------------
Connection Index : 0x01 (Port 1)
Connection Status : 0x01 (DeviceConnected)
Current Config Value : 0x01 (Configuration 1)
Device Address : 0x3D (61)
Is Hub : 0x00 (no)
Device Bus Speed : 0x01 (Full-Speed)
Number Of Open Pipes : 0x02 (2 pipes to data endpoints)
Pipe[0] : EndpointID=2 Direction=IN ScheduleOffset=0 Type=Interrupt wMaxPacketSize=0x20 bInterval=1 -> 420 Bits/ms = 52500 Bytes/s
Pipe[1] : EndpointID=2 Direction=OUT ScheduleOffset=0 Type=Interrupt wMaxPacketSize=0x20 bInterval=1 -> 420 Bits/ms = 52500 Bytes/s
...
------------------- HID Descriptor --------------------
bLength : 0x09 (9 bytes)
bDescriptorType : 0x21 (HID Descriptor)
bcdHID : 0x0100 (HID Version 1.00)
bCountryCode : 0x00 (00 = not localized)
bNumDescriptors : 0x01
Data (HexDump) : 09 21 00 01 00 01 22 25 00 .!...."%.
Descriptor 1:
bDescriptorType : 0x22 (Class=Report)
wDescriptorLength : 0x0025 (37 bytes)
....Okay, so that’s just something that names itself an “HID” device (perhaps, to avoid the requirement to install any drivers into users’ system), with two endpoints (send and receive). The device is USB2 (so why is a USB3 port recommended? I even bought the USB3 extender, which is 5x more expensive…) and just a HID, not a keyboard, not a mouse — just a “vendor-defined device”. Good.
The next obvious step — let’s download the Wireshark and install it including all the useful modules, especially USBPcap.
Run it… Oh, I have five USBPcap devices visible (same as the number of the root host controllers). Okay, repeat the on/off cable game: disconnect the cable, start capture on 1st USBPcap device, connect cable — do we see new packets arrive? No, repeat with the 2nd. Okay, found the required endpoint, great, as we connected the cable and capture — run the gateway, wait until it shows that everything is connected. Spin the platform’s backplate. Moves shoes a bit on a platform. Stop capture.
Time to filter out uninteresting parts. In my system, I saw the packets come from 5.9.0 — so enter into the filter field `usb.addr == “5.9.0”`. Hmmm. Now I only see DESCRIPTOR packets and nothing else. Oh, right! I need packets from other endpoints too, so change the filter to `usb.addr ~ “5.9.”`. Nice! Now I see only communication with the platform.
`GET DESCRIPTOR`, `SET CONFIGURATION`… Not really interesting… Scroll down… Ahha! `URB_INTERRUPT out`:
And little below, paired with `URB_INTERRUPT in`. Great, but need to make it more comfortable to stare at them. Let’s select the “HID Data” line in the packet contents pane at the bottom of the wireshak, do the right click => “Apply as Column”, and then drag the columns to change the order. That’s much better:
Here we clearly see that all the packets have the same size of 59 bytes, all of them are URB_INTERRUPT with 32 bytes of actual data inside.
Instantly obvious that the packet has a fixed structure:
[1F] [55] [AA] [00] [00] ...And I can’t see anything that looks like a checksum. But as USB guarantees the delivery for Interrupt packets (and packet orders as well), we don’t need any checksum.
Okay, time to look one more time, now in a sequence:
(out) [1F] [55] [AA] [00] [00] [31] [ zeros ]
(in) silence
(out) [1F] [55] [AA] [00] [00] [05] [ zeros ]
(in) [1F] [55] [AA] [00] [00] [05] [00] [03] [ zeros ]
(out) [1F] [55] [AA] [00] [00] [21] [ zeros ]
(in) [1f] [55] [AA] [00] [00] [21] [00] [03] [ca] [f8] ... (more data here)
(out) [1F] [55] [AA] [00] [00] [A0] [00] [02] [ zeros ]
(in) silence
(out) [1F] [55] [AA] [00] [00] [31] [ zeros ]
(in) silence
(out) [1F] [55] [AA] [00] [00] [30] [ zeros ]
(in) silence
(out) [1F] [55] [AA] [00] [00] [30] [ zeros ]
(in) silence
(out) [1F] [55] [AA] [00] [00] [30] [ zeros ]
(in) [1F] [55] [AA] [00] [00] [33] [00] [01] [00] [09] [00] [64] [ zeros ]
(in) [1F] [55] [AA] [00] [00] [33] [00] [02] [00] [09] [00] [64] [ zeros ]
(in) [1F] [55] [AA] [00] [00] [33] [00] [00] [00] [09] [00] [64] [ zeros ]
(in) [1F] [55] [AA] [00] [00] [30] [01] [01] [00] [00] [c4] [00] [ff] [ff] [ff] [ff] [ff] [ff] [ff] [ff] [ff] [00] [00] [00] [00] [00] [82] [00] [00] [00] [00] [00]
...
1f55aa000030020100006f00ffffffffffffffffff0000000000780000000000
1f55aa000030020100006f00ffffffffffffffffff0000000000780000000000
1f55aa00003001010000c400ffffffffffffffffff0000000000800000000000
1f55aa000030020100006f00ffffffffffffffffff0000000000760000000000
...
1f55aa00003000e8000bd38a2d7e00000000000000ffffff0000000000000000
1f55aa00003001010000c400ffffffffffffffffff0000000000820000000000
1f55aa000030020100006f00ffffffffffffffffff0000000000760000000000
1f55aa00003000e8000bd38a2d7e00000000000000ffffff0000000000000000
1f55aa00003001010000c400ffffffffffffffffff0000000000810000000000
1f55aa000030020100006f00ffffffffffffffffff0000000000780000000000
1f55aa00003000e8000bd38a2d7e00000000000000ffffff0000000000000000
1f55aa00003001010000c400ffffffffffffffffff0000000000810000000000
1f55aa000030020100006f00ffffffffffffffffff0000000000770000000000
1f55aa00003000e8000bd38a2d7e00000000000000ffffff0000000000000000Great! Out of this stream, we can figure out that the packet structure is:
- [1F] => looks like the packet’s data length since all the packets have the same length — it’s always 0x1F
- [55] [AA] => standard bit sequence 10101010 and 01010101 to synchronize UART clocks
- [00] [00] => couple of zero bytes (to let the UARTs rest/reset?)
- [XX] => afterward goes single byte the command/answer ID
- The rest of the packet — the command parameters and answers data.
On top of that, we can see that commands [05] and [21] read some configuration. [A0] adjusts the brightness of the backlight (I also confirmed that by adjusting the slider in the gateways’ settings). And, obviously, the command [30] starts the data updates stream. Stream starts with [33] with some settings, and then the stream of [30]s comes with feet data and the backplate angle.
So we’ve got an idea of what we dealing with. Let’s look again inside the gateway, anything related to HID?
//...
[DllImport("KATDeviceSDK.dll")]
public static extern void GetSensorInformation(
out KATSDKInterfaceHelper.sensorInformation obj,
string sn);
[DllImport("KATDeviceSDK.dll")]
[return: MarshalAs(UnmanagedType.I1)]
public static extern bool DeepSleep(string sn, int type);
[DllImport("KATDeviceSDK.dll")]
[return: MarshalAs(UnmanagedType.I1)]
public static extern bool WriteDeviceId(string sn, int id);
[DllImport("KATDeviceSDK.dll")]
[return: MarshalAs(UnmanagedType.I1)]
public static extern bool ReadDeviceId(string sn, out int id);
[DllImport("KATDeviceSDK.dll")]
[return: MarshalAs(UnmanagedType.I1)]
public static extern bool SendHIDCommand(
string sn,
byte[] command,
int cmdlen,
byte[] outBuffer,
int outLen);
//...Great, the actual handling is inside of the native code in `KATDeviceSDK.dll`.
However, looking at usages of `SendHIDCommand`, like there:
public static int Get_Version(string sn)
{
byte[] numArray1 = new byte[32];
byte[] numArray2 = new byte[32];
numArray1[0] = (byte) 32;
numArray1[1] = (byte) 31;
numArray1[2] = (byte) 85;
numArray1[3] = (byte) 170;
numArray1[4] = (byte) 0;
numArray1[5] = (byte) 0;
numArray1[6] = (byte) 5;
for (int index = 0; index < 3; ++index)
{
if (KATSDKInterfaceHelper.SendHIDCommand(sn, numArray1, ((IEnumerable<byte>) numArray1).Count<byte>(), numArray2, ((IEnumerable<byte>) numArray2).Count<byte>()))
{
if (numArray2[0] == (byte) 0 && numArray2[1] == (byte) 0 && numArray2[2] == (byte) 0)
{
C2FirmwareUpdaeManager.SetFirmwareUpdaeState(sn, C2FirmwareUpdaeManager.nowVersion);
C2FirmwareUpdaeManager.nowVersion = -1;
}
else if (numArray2[0] == (byte) 31 && numArray2[1] == (byte) 85 && numArray2[2] == (byte) 170 && numArray2[5] == (byte) 5)
{
C2FirmwareUpdaeManager.nowVersion = (int) numArray2[7];
C2FirmwareUpdaeManager.SetFirmwareUpdaeState(sn, C2FirmwareUpdaeManager.nowVersion);
return C2FirmwareUpdaeManager.nowVersion;
}
}
Thread.Sleep(10);
}
C2FirmwareUpdaeManager.nowVersion = -1;
C2FirmwareUpdaeManager.SetFirmwareUpdaeState(sn, C2FirmwareUpdaeManager.nowVersion);
return C2FirmwareUpdaeManager.nowVersion;
}We can learn more commands:
- command [05] — reads the version of the device
- command [07] — `Set_SN`, changes serial number?
- command [A0] — not only responsible for the backlight but also haptic vibration control for C2+
- command [21] — reads pairing data with the sensors (3 sensors, each sensor address is 6 bytes)
Additionally, we learn that Vehicle Hub (the seat) works using similar packets.
Let’s record all current findings into some file and switch from dotPeek into IDA, time to look into `KATDeviceSDK.dll`.
Quickly skim through all the exported functions (with adding some comments and rename variables / common functions etc, the usual):
char __fastcall ReadDeviceId(char *a1, byte *a2, __int64 a3)
{
// ...
KatBySN = (HANDLE **)FindKatBySN(a1, a2, a3, a1);
v5 = (__int64 *)KatBySN;
if ( !KatBySN )
return 0;
v6 = *KatBySN;
memset(v16, 0, sizeof(v16));
if ( (int)hid_read_timeout(v6, (byte *)v16, 0x20ui64, 100) < 0 )
return 0;
v11 = 0xAA551F20;
v12 = 0;
v13 = 3;
v14 = 0i64;
v15 = 0i64;
hid_write(*v5, (char *)&v11, 0x1Fui64);
timeout = hid_read_timeout((HANDLE *)*v5, (byte *)v16, 0x20ui64, 100);
v8 = (HANDLE *)*v5;
v9 = timeout;
if ( v8 )
{
CancelIo(*v8);
CloseHandle(v8[9]);
CloseHandle(*v8);
LocalFree(v8[3]);
free(v8[5]);
free(v8);
}
if ( v9 < 0 )
return 0;
*(_DWORD *)a2 = (unsigned __int8)v16[7];
return 1;
}From all of the functions like these, we learn more:
- [23] — put the device to sleep
- [08] — reads sensor ID (the 6 bytes, used for pairing)
- [31] — MCUStopSend. So the [31] stops the stream, while [30] starts one.
- [03] — ReadDeviceID, hmm… what kind of ID? don’t know yet, will see
- [04] — WriteDeviceID
- [20] — WriteSensorPair, yes, and the [21] we saw above then reads the pairing data.
Great, but we still don’t know the meaning of answers [30] (and [33])… We have there also one more function `StartListen` which, by the naming, should start stream reading. What do we have inside of it?
// ...
if ( ((v13 - 12087) & 0xFFFFEFFF) == 0 )
{
v18[1] = (__crt_strtox *)'lld.2Cr';
v19 = 15i64;
goto LABEL_32;
}
switch ( v13 )
{
case 12070:
strcpy((char *)&v18[1], "rC.dll");
v19 = 14i64;
LABEL_32:
v18[0] = (__crt_strtox *)'evirDTAK';
goto LABEL_33;
case 12053:
case 12069:
case 12176:
sub_7FF8E66F7000(v18, 18i64, v3, "KATDriverLocoS.dll");
goto LABEL_33;
case 3855:
sub_7FF8E66F7000(v18, 21i64, v3, "KATDriverWalkMini.dll");
LABEL_33:
v14 = (__crt_strtox *)v18;
if ( v20 >= 0x10 )
v14 = v18[0];
__crt_strtox::multiply_by_power_of_ten(v14, a1, (unsigned int)v3);
goto LABEL_36;
case 40741:
sub_7FF8E66F7000(v18, 16i64, v3, "KATDriver3DT.dll");
goto LABEL_33;
}
sub_7FF8E66F1090("Device Not Implement!\n");Oh my, that’s awesome optimizations of inlines and small std::string’lets! Although, the idea is clearly visible. We load `KATDriver${type}.dll` and translate the call there.
Stop… What are these `case` numbers?… Let’s change them into hex:
if ( ((v13 - 0x2F37) & 0xFFFFEFFF) == 0 )
{
v18[1] = (__crt_strtox *)'lld.2Cr';
v19 = 15i64;
goto LABEL_32;
}
switch ( v13 )
{
case 0x2F26:
strcpy((char *)&v18[1], "rC.dll");
v19 = 14i64;
...Cool! Indeed, that PID of our device — 0x2F37 for the C2 platform. Great, although, not important right now. Let’s load `KATDriverC2.dll` into IDA.
Inside we have exactly the same hid_* functions, as in other dll, plus `LED()`, `Vibrate()`, and `StartListen()`/`StopListen()`.
The `StartListen` function is something huge and long, but it has lots of debug information inside which helps to speed up reading.
Quickly skim through decompiled logic:
- Load `KatWalkerBase.dll` and dynamically import multiple functions from it (`InitAlgorithm`, `ShutDownAlgorithm`, `UpdateIMU`, `SensorDataUpdated`, `UpdateExtension`, `UpdateOpticalSensor`).
- Connecting to the device (open HID device file)
- Get the type of the device:
if ( v65 == 0x2F37 )
{
v66 = "KATVR Walk Coord2";
v67 = 17i64;
}
else
{
if ( v65 != 0x3F37 )
{
sub_180001050("Invaild Device! 0x%x\n", v65);
goto LABEL_321;
}
v66 = "KATVR Walk Coord2 Core";
v67 = 22i64;
}Ah, means C2 and C2Core use different USB PIDs… That also explains the `(v13–0x2F37) & 0xFFFFEFFF)` from the code inside of `KATDeviceSDK`.
Once we get a connection, we send the initial sequence:
- Send command [31]
- Sleep for 1s
- Send command [30]
- Open named shared memory `KAT_DEVICE_CONNECTION_`
- Start infinity loop of read-process.
Great, that is exactly what we looked for! Inside of this huge read-process loop we have logic to handle errors, disconnects, and silence. Irrelevant for now, though. Scroll down… down… down… Ahha! Here is what we need:
if ( cmdPtr[4] == 0x33 )
{
if ( ansLen < 11 )
goto LABEL_277;
sub_1800063F0(&Buf2);
ansLen = *(&v250[64] + 4);
}
if ( cmdPtr[4] == 0x32 )
{
// ...That means, that an answer packet marked with [33]… Just ignored. Well, legacy, perhaps?
Packet [32] contains sensor configuration and its charge status: ID (that’s what it means under the ID above!), firmware version, charge level and charging status. The type of the sensor here is fixed (1 == backplate, 2 == left foot, 3 == right foot).
Packet [30] contains an update for the sensor data, and it uses the comparison of the sensor with the ID reported in packet [32] (and configured during a paring procedure via WriteDeviceId).
The data update processing for the feet is trivial:
float __fastcall parseFootSensor(char *cmdBuf, FOOT_SENSOR_DATA *sensorDataOut, int sensorPacketNo)
{
//...
sensorDataOut->packetNo = sensorPacketNo;
timestamp_us = sensorDataOut->timestamp_us;
v8.x = *(cmdBuf + 10) / 59055.117;
v8.y = *(cmdBuf + 11) / 59055.117;
sensorDataOut->vec = v8;
sensorDataOut->status = cmdBuf[25];
sensorDataOut->something = *(cmdBuf + 4);
result = -5.1896949e11;
current_us = Xtime_get_ticks() / 10000000.0;
sensorDataOut->timestamp_us = current_us;
v7 = current_us - timestamp_us;
sensorDataOut->timedelta_us = v7;
if ( v7 < 0.004 )
sensorDataOut->timedelta_us = 0.004;
return result;
}On the contrary, the backplate/direction sensor processing is far less trivial:
__int64 __fastcall parseDirection(char *cmdBuf, _DIR_SENSOR_DATA *sensorDataOut)
{
// It starts with simple and trivial unpacking:
sensorDataOut->packet_no = dir_packet_count;
v4 = COERCE_UNSIGNED_INT(-*(cmdBuf + 7));
v4.m128_f32[0] = v4.m128_f32[0] * 0.00390625;
v5 = COERCE_UNSIGNED_INT(*(cmdBuf + 9));
v5.m128_f32[0] = v5.m128_f32[0] * 0.00390625;
v6 = *(cmdBuf + 8) * 0.00390625;
*&sensorDataOut->in_vel.X = _mm_unpacklo_ps(v4, v5).m128_u64[0];
sensorDataOut->in_vel.Z = v6;
qq2 = pow(2.0, -14.0) * *(cmdBuf + 4);
qq3 = pow(2.0, -14.0) * *(cmdBuf + 5);
qq4 = pow(2.0, -14.0) * *(cmdBuf + 6);
qq1 = pow(2.0, -14.0) * *(cmdBuf + 3);
sensorDataOut->in_dir.X = qq1;
sensorDataOut->in_dir.Y = qq2;
sensorDataOut->in_dir.Z = qq3;
sensorDataOut->in_dir.W = qq4;
// But then inline quarts processing plus XMM handling makes it hard to follow:
if ( 0.0 > 2.0 )
sqrt2 = sqrtf(2.0);
else
sqrt2 = fsqrt(2.0);
sin45 = -(sqrt2 * 0.5);
v13 = ((sin45 * sin45) + 0.0) + ((sin45 * sin45) + 0.0);// 1
VERTICAL.X = -0.0 / v13; // 0
VERTICAL.Y = -sin45 / v13; // 1/sqrt(2)
VERTICAL.Z = -0.0 / v13; // 0
VERTICAL.W = sin45 / v13; // -1/sqrt(2)
quaternion_multiply(&a1, &VERTICAL, &sensorDataOut->in_dir);
v14 = sinf(0.78539819); // sin(45deg)=1/sqrt(2)
*&v15 = 0x3F490FDBu;
*&v15 = cosf(0.78539819); // cos(45)=1/sqrt(2)
v16 = *&v15;
__zero_0 = (v14 * 0.0) * a1.Z;
__zero_1 = (v14 * 0.0) * a1.X;
__zero_2 = (v14 * 0.0) * a1.Y;
__zero_3 = (v14 * 0.0) * a1.W;
v16.m128_f32[0] = (((*&v15 * a1.W) - __zero_1) - __zero_2) - (v14 * a1.Z);// v16 = (v16, 0, 0, 0)
v21 = _mm_shuffle_ps(v16, v16, 0); // v21 = (v16, v16, v16, v16)
v21.m128_f32[0] = (((*&v15 * a1.X) + __zero_3) + __zero_0) - (v14 * a1.Y);// v21 = (v21, v16, v16, v16)
v22 = _mm_shuffle_ps(v21, v21, 225); // v22 = (v16, v21, v16, v16)
v22.m128_f32[0] = (((*&v15 * a1.Y) + __zero_3) + (v14 * a1.X)) - __zero_0;// v22 = (v22, v21, v16, v16)
v23 = _mm_shuffle_ps(v22, v22, 198); // // v23 = (v16, v21, v22, v16)
v23.m128_f32[0] = (((*&v15 * a1.Z) + (v14 * a1.W)) + __zero_2) - __zero_1;// // v23 = (v23, v21, v22, v16)
sensorDataOut->in_dir = _mm_shuffle_ps(v23, v23, 201);// res = (v21, v22, v23, v16)
Y = sensorDataOut->in_dir.Y;
v25 = -sensorDataOut->in_dir.Z;
W = sensorDataOut->in_dir.W;
sensorDataOut->in_dir.X = -sensorDataOut->in_dir.X;// -v21 => -a1.x/sqrt(2) + a1.y/sqrt(2)
sensorDataOut->in_dir.Y = Y; // +v22 => a1.y/sqrt(2) + a1.x/sqrt(2)
sensorDataOut->in_dir.Z = v25; // -v22 => -a1.z/sqrt(2) - a1.w/sqrt(2)
sensorDataOut->in_dir.W = W; // +v16 => a1.w/sqrt(2) - a1.z/sqrt(2)
// The rest is simple again:
timestamp_us = sensorDataOut->timestamp_us;
*&sensorDataOut->field_1C = _mm_unpacklo_ps(0i64, 0i64).m128_u64[0];
*&sensorDataOut->field_34 = _mm_unpacklo_ps(0i64, 0i64).m128_u64[0];
sensorDataOut->field_3C = 0;
sensorDataOut->field_24 = 0;
v15 = Xtime_get_ticks() / 10000000.0;
sensorDataOut->timestamp_us = v15;
sensorDataOut->timedelta_us = v15 - timestamp_us;
result = cmdBuf[24] >> 7;
sensorDataOut->gap60 = result;
return result;
}As you can see, I have to fall back to add comments throughout to keep track of what is where. By fully comprehending this large bit I’ve got this one:
out.x = in.x/sqrt(2) + in.y/sqrt(2);
out.y = in.y/sqrt(2) + in.x/sqrt(2);
out.z = -in.z/sqrt(2) - in.w/sqrt(2);
out.w = in.w/sqrt(2) - in.z/sqrt(2);I believe that’s just the normalization of the received data, but I didn’t get down at the time to the bottom of it — just wrote it down into the notepad and moved on.
Further, the code checks number of correct packets seen and acts when it is less than 40 packets per sensor per second: in that case it sends [31] and [30] again to reset the connection stream.
Once the data is processed, it sends the data to the `UpdateOpticalSensor` and other functions from KatWalkerBase into “Algorithm”. Anyway, that’s not important for us again for now. So, let’s consult with the notepad on what we’ve learned in the end:
Kat Walk C2 Receiver USB HID protocol:
HID Vendor 0xC4F4, PID 0x2F37 for C2, 0x3F37 for C2Core, 0x8F37 for C2+ Seat receiver
Command format:
prefix: 0x1F 0x55 0xAA 0x00 0x00 {command} {arguments, 32bytes total}
31 85 170 0 0
Command:
GetFirmwareVersion {Command=0x05}: {no args}
SetSN {Command=0x07}: {Args = Serial number as ascii string}
CloseVibration {Command=0xA0}: 0x00 0x02 0x00 x00
SetLED {Command=0xA1}: 0x01 0x02 {HIGH(LED * 1000)} {LOW(LED*1000)}
SetVibration {Command=0xA1}: 0x00 0x02 {HIGH(Vibr*1000)} {LOW(Vibr*1000)}
StartRead {Command=0x30}: {no args, start listen to updates}
StopRead {Command=0x31}: {no args, stop listen to updates}
WritePairing {Command=0x20}: Cnt 0x00 0x00 {Mac0} ... {MacN} // Write $Cnt MACs of sensors, 6 bytes each
ReadPairing {Command=0x21}: Cnt 0x00 0x00 {Mac0} ... {MacN} // Write $Cnt MACs of sensors, 6 bytes each
GetSensorInformation{Command=0x08}: {no args} // Read MAC of device (receiver/sensor on usb)
ReadDeviceId {Command=0x03}: {no args} // Read ID of device (receiver/sensor on usb) anwser in byte 7 (offset 5)
DeepSleep {Command=0x23}: SleepType 0 0 0 0 // SleepType==0 => off; 1=>on (put to sleep? or disable sleep? dunno)
WriteDeviceId {Command=0x04}: ID 0 0 0 // Write ID of device (via USB; left/right/body etc}
Answer format:
prefix: 0x1F 0x0x55 0xAA 0x00 0x00 {Command No}
GetSN {Command=0x06}: 0x?? {Version}
Stream Updates:
len 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
30: main update? expected len >= 26
0000 1f 55 aa 00 00 30 01 01 00 00 c4 00 ff ff ff ff ff ff ff ff ff 00 00 00 00 00 82 00 00 00 00 00
0000 1f 55 aa 00 00 30 01 01 00 00 c4 00 ff ff ff ff ff ff ff ff ff 00 00 00 00 00 80 00 00 00 00 00
0000 1f 55 aa 00 00 30 01 01 00 00 c4 00 ff ff ff ff ff ff ff ff ff 00 00 00 00 00 81 00 00 00 00 00
0000 1f 55 aa 00 00 30 02 01 00 00 6f 00 ff ff ff ff ff ff ff ff ff 00 00 00 00 00 78 00 00 00 00 00
0000 1f 55 aa 00 00 30 02 01 00 00 6f 00 ff ff ff ff ff ff ff ff ff 00 00 00 00 00 76 00 00 00 00 00
0000 1f 55 aa 00 00 30 02 01 00 00 6f 00 ff ff ff ff ff ff ff ff ff 00 00 00 00 00 78 00 00 00 00 00
feet sensors:
^^ [5] sensor id
^^^^^ [8-9, short => HZ] ??
^^^^^ [20-21, short => X] Speed X/59055.117
^^^^^ [22-23, short => Y] Speed Y/59055.117
^^ [25, byte] status?
len 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
30: ?!??! main update? expected len >= 26
0000 1f 55 aa 00 00 30 00 e8 00 0b d3 8a 2d 7e 00 00 00 00 00 00 00 ff ff ff 00 00 00 00 00 00 00 00
^^ [5] sensor id
direction sensor: Q1-Q4 is source for quarterion, A/B/C at the end -- dunno (yet)
^^^^^ [6-7, short => Q1] (2^-14) * Q1
^^^^^ [8-9, short => Q2] (2^-14) * Q2
^^^^^ [10-11, short => Q3] (2^-14) * Q3
^^^^^ [12-13, short => Q4] (2^-14) * Q4
^^^^^ [14-15, short => A] (-A) * (1/256)
^^^^^ [16-17, short => C] (+C) * (1/256)
^^^^^ [18-19, short => B] (+B) * (1/256)
^^ [24, bool] bit7: button pressed status
len 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
32: ?!??! configuration/state? expecteed len >= 11
0000 1f 55 aa 00 00 32 00 01 02 00 64 04 00 00 00 00 00 00 00 ff ff ff 00 00 00 00 00 00 00 00 00 00
0000 1f 55 aa 00 00 32 01 02 02 00 64 05 ff ff ff ff ff ff ff 00 00 00 00 00 81 00 00 00 00 00 00 00
0000 1f 55 aa 00 00 32 02 03 02 00 64 05 ff ff ff ff ff ff ff 00 00 00 00 00 75 00 00 00 00 00 00 00
^^ [10] firmware version
^^^^^ [8-9, short] charge level
^^ [7] bit 0: connected; bit 1: ?
^^ [6] sensor type: 1 == direction; 2 == left; 3==right
^^ [5] sensor id
33: ?!??! [KATDriverC2 ignores packets, expects ansLen >= 11]
0000 1f 55 aa 00 00 33 00 00 00 09 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000 1f 55 aa 00 00 33 00 01 00 09 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000 1f 55 aa 00 00 33 00 02 00 09 00 64 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00Native Gateway — isn’t it easy!
Now we know how to communicate with the treadmill, but what we can do with this knowledge? We can write our own SDK, with our own lunar module!
Why? Because we can then embed it into the game. Yes, I don’t write games, but in the community, several people do. For example, Utopia Machina builds a game that is focused on the treadmill to utilize its abilities for physical exercise (to maximize walking and running time). And he had a question — do we really need the Gateway, especially, for the Standalone Game?
Since I’ve learned how to get data without the gateway, I’ve decided to see if I can get the data from the treadmill directly, right from the headset.
Since I never wrote Android apps (tiny kernel drivers don’t count) — it’s definitely interesting to try!
# Prototype: facelift for random project and setup the environment
Obviously, before building my own app, I need to learn how to do so. The easiest way to do so — dissect some other app. Plus I need to validate my ideas and check the communication before I learn more than required for that. That said, I’ve found USBHIDTerminal which looks like a good starting point. The problem is its outdated, and thus needs some facelift, dependencies upgrade, etc.
Let’s do so! First step: get the Android Studio.
Then, facelift itself. For others, who also never done that before, here are the hints and pointers:
- Choose your target Android SDK. The app uses SDK 22, which is very outdated. Quest 3 as of today runs Android 12L which requires SDK 32. My phone, running Android 14, requires SDK 34. So, as we install Studio don’t forget to install SDK 32 as well — we’ll need it.
- Whenever you open an outdated project (and I’ve seen that a couple of times), Android Studio sometimes offers an automatic upgrade script — that’s a great option; sometimes don’t. I’ve agreed to everything it recommends to upgrade automatically, as it saves time.
- Then upgrade gradle if it wasn’t offered by the studio: go to `build.gradle` and change “3.6.3” to “8.2.1” (or whatever version it offers and is up to date right now). You don’t need the freshest version, just “fresh enough” for Studio to work. After the change, trigger project sync with gradle (“Sync Project With Gradle Files” icon of elephant in the right top corner of the screen, Ctrl+Shift+O). After this point, Studio starts processing the files and shows you multitude of errors and warnings.
- Change the minimum and target SDK version (still inside of the `build.gradle`). Since I target to the headset, I’ve set compileSdkVersion to 32, minSdkVersion to 32, and targetSdkVersion to (you won’t believe it!) 32.
- Then we should “migrate” to the fresher `androidx` version (for even older projects, you may actually need to migrate from the thing that was before to it). In this case, it just means to change inside of the `build.gradle`:
- * `implementation ‘androidx.legacy:legacy-support-v4:1.0.0’` into `implementation ‘androidx.appcompat:appcompat:1.0.2’`
- * inside of `android` section add the line `namespace “com.appspot.usbhidterminal”`
- For convenience, change “JavaVersion.VERSION_1_8” to “JavaVersion.VERSION_17”
- And update `AndroidManifest.xml`:
- * Remove `targetSdkVersion` (it moved to `build.gradle`)
- * remove unused `uses-permissions`,
- * adds to each `<activity/>` now mandatory attribute `android:exported=”true”`.
- I’ve also removed dependencies to `httpd` since I don’t need it for my experiments.
After a few cycles of attempts to build — fix errors — repeat, the project got built and even runs in the simulator. At this point we can connect the cable from the PC to the Phone — and trivially run the app on the phone — and it even runs! And if we connect the platform with USB — app shows it in the selection menu.
# Prototype: let’s peek at how to write applications
As I’ve said already, I’ve never written Android Apps before, so I’ve taken a deep look at the USBHIDTerminal. It quickly became clear, that there was nothing new: the UI build up quite similar to GTK+, and that one I know.
So, `res/layout/$Activity.xml` contains GUI description — which components lie inside of which other components. 1-to-1 matching the glade. Components has signals, which we listen and do something. Great, let’s add a button:
<Button
android:id="@+id/btnLedOn"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_toRightOf="@id/btnClear"
android:minHeight="36dip"
android:text="Led ON"
android:textAppearance="?android:attr/textAppearanceSmallInverse" />Actually, lets make it 4 buttons (…). Then we go to the main application code and add some boilerplate:
public class USBHIDTerminal extends Activity implements View.OnClickListener {
// ...
private Button btnInit;
private Button btnStop;
private Button btnLedOn;
private Button btnLedOff;
// ...
private void initUI() {
// ...
btnLedOn = (Button) findViewById(R.id.btnLedOn);
btnLedOn.setOnClickListener(this);
// ... three more
}
public void onClick(View v) {
// ...
} else if (v == btnInit) {
eventBus.post(new USBDataSendTypedEvent("0x1F 0x55 0xAA 0x00 0x00 0x30 0x00 0x00 0x00 0x00", true));
} else if (v == btnStop) {
eventBus.post(new USBDataSendTypedEvent("0x1F 0x55 0xAA 0x00 0x00 0x31 0x00 0x00 0x00 0x00", true));
} else if (v == btnLedOn) {
eventBus.post(new USBDataSendTypedEvent("0x1F 0x55 0xAA 0x00 0x00 0xA1 0x01 0x02 0x03 0x8F", true));
} else if (v == btnLedOff) {
eventBus.post(new USBDataSendTypedEvent("0x1F 0x55 0xAA 0x00 0x00 0xA1 0x01 0x02 0x00 0x00", true));
}
// ...
}
public void onEvent(DeviceAttachedEvent event) {
// ...
btnLedOn.setEnabled(true);
btnLedOff.setEnabled(true);
btnStop.setEnabled(true);
btnInit.setEnabled(true);
// ...
// and similar into DeviceDetachedEvent
}Build application again, run in the simulator — see out buttons, but we can’t connect to the device from it. Sad… Is there any way to make the USB device visible inside? Quick googling didn’t give me the answer. Deeper search didn’t help either. Sad :(
Well, that means we’ll continue testing on the phone. Connect the PC cable, install the app, disconnect the PC, connect the treadmill cable, run, and enjoy! We can turn the LED on and off! And we can run the stream — everything seems to work as we want.
Although, the HID Terminal is not something useful to test, so, let’s build out our application.
# Prototype: building everything from scratch
To build our app we can create a new application in the Android Studio, and enable the latest features, including Kotlin (since I never wrote on Kotlin but heard a lot of good about it).
For the UI we want to build something gateway-like: a direction arrow plus two fields for the foot movement vectors.
That means we need two components: one for the feet and one for the arrow. So, file=>create=>new=>component, `ArrowView`.
The template is ready to use and shows everything we want to know: how to introduce parameters, how to make getters and setters in Kotlin, syntax, constructor and ready to use drawing function. That makes it trivial to rename, edit, copy-paste, and so on. For the arrow we need:
- arrow angle property,
- text to draw on the component (so we can compute its width to draw it in the middle),
- component parameters (width-height as interface is kind of elastic).
And we have examples for everything, including basic onDraw. Oh, looking at it… It resembles the drawing method used in GTK based on cairo. So, our drawing code idea is simple: draw a thin vertical rectangle as a base, then apply the rotation matrix to one side, draw another small rectangle; apply the rotation matrix to the other side and draw another rectangle, so we get something arrow-shaped.
Then add an overall rotation matrix before all of that — and we’ve got it done. We can keep the text output the same as it was before.
Similarly, we can create a second component to draw the feet dots instead of an arrow…
And then we swear a little bit since the Studio starts complaining about attributes clashing. To fix that we need to fix `res/values/attrs.xml`. To learn how to fix it properly we need to do some googling. It’s a bit weird that template creation is not working to create two components in a row, but anyway, that’s how the fixed version looks like:
<resources>
<attr name="text" format="string" />
<attr name="textSize" format="dimension" />
<attr name="textColor" format="color" />
<declare-styleable name="FeetDotView">
<attr name="text" />
<attr name="textSize" />
<attr name="textColor" />
</declare-styleable>
<declare-styleable name="ArrowView">
<attr name="textSize" />
<attr name="textColor" />
</declare-styleable>
</resources>As long as the type of the attribute is the same, we can group them there as we want.
The next step is to edit `res/layout/activity_main.xml`, where we need to add a ~~VBox with a couple of HBox inside~~, err, I mean to add an `<LinearLayout android:orientation=”vertical”>` and place inside of it two `<LinearLayout android:orientation=”horizontal”>`.
Inside the first horizontal layout we add the control buttons like before, inside the second HBox we put ArrowView and two FeetDotViews.
Now, time to think about the architecture of the whole application. What do we need?
- We need a background processing thread to handle a USB connection, it will receive and send packets.
- We need a driver for the VR Treadmill itself, which will turn USB packets into meaningful data.
- We’ll need a main thread, that will run the background processing and will then update UI elements.
- And, of course, we’ll need something that will work as a middleware bus between background and UI threads. We have options on how to achieve that: we can either poll the latest data periodically (like, every drawing frame) or use something to send the update every time we’ve got a fresh update.
Seems trivial, and USBHIDTerminal works in a similar way, so dissecting it we learn that background activity threads in the android are made using
“Service”, and there is a very convenient and simple library [eventbus](https://greenrobot.org/eventbus/). Note: Thread() is still necessary, as
Service is only groups non-UI application activity but still executed on the main thread. Of course, I will definitely read one day the documentation if I ever need to write an Android application for real, not as a prototype.
Seems trivial, and USBHIDTerminal works in a similar way, so dissecting it we learn that background activity threads in the android are made using “Service”, and there is a very convenient and simple library eventBus. Note: Thread() is still necessary, as Service is only groups non-UI application activity but still executed on the main thread. Of course, I will definitely read one day the documentation if I ever need to write an Android application for real, not as a prototype.
So, import the latest `eventbus` into the application and via File=>New=>Service create `KatGatewayService`.
The service should do two things: find the USB device connected (and/or catch the moment when the cable is attached) and then communicate over it.
To find the cable we should USB Host feature, which we should add into `AndroidManifest.xml` via `<uses-feature android:name=”android.hardware.usb.host” />`. To detect connection/disconnection of the device we should add to our MainActivity corresponding broadcast events filters and include meta-data with a list of the devices we are interested in:
<activity
android:name=".MainActivity"
android:exported="true"
android:screenOrientation="landscape">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<action android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED" /> <!-- !!!!! -->
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<meta-data
android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED"
android:resource="@xml/usb_device_filter" /> <!-- !!!!! -->
</activity>The `xml/usb_device_filter.xml` file is just a list of the devices:
<?xml version="1.0" encoding="utf-8"?>
<resources>
<!-- in hex: USB\VID_C4F4&PID_3F37 = C2 Core Receiver -->
<usb-device vendor-id="50420" product-id="16183" />
<!-- in hex: USB\VID_C4F4&PID_2F37 = C2/C2+ Receiver -->
<usb-device vendor-id="50420" product-id="12087" />
</resources>By providing the list of the USB devices for the filter we not only allow our application to start upon connection but also enable an additional checkbox “always allow to connect to this device” to the permissions checking popup.
Well, now we add to the `class MainActivity` initialization of eventBus and run the USB Service we’ve created:
class MainActivity : AppCompatActivity(), View.OnClickListener {
protected var eventBus = EventBus.getDefault()
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
eventBus.register(this)
katGatewayService = Intent(this, KatGatewayService::class.java)
startService(katGatewayService)
setContentView(R.layout.activity_main)
initUI()
}
//...We already know what to do inside of the `initUI` and how to write `onClick` is also trivial. The new EventBus version switched from the use of fixed `onEvent` handlers to decorators, which means we just do `onWhaterverWeNeed` and add `@Subscribe{}` decorator to it.
Done, the UI and main thread are ready, time to construct the background service.
The service should subscribe to USB events and ask for the permission to access the device. Android permissions system works by deferring events upon permissions change (sent whenever the user confirms/rejects them from system-shown UI) plus functions to check if the permission is available or not.
That means all the permissions-related code needs to be split into the request and the reaction in the different handlers.
So, upon creation, the service will set up the intent filter (aka “subscribe to events” in old-school terminology) and request permission:
override fun onCreate() {
super.onCreate()
usbManager = getSystemService(USB_SERVICE) as UsbManager
permissionIntent = PendingIntent.getBroadcast(
this,
0,
Intent(ACTION_USB_PERMISSION),
PendingIntent.FLAG_MUTABLE
)
filter = IntentFilter(ACTION_USB_PERMISSION)
filter.addAction(UsbManager.ACTION_USB_DEVICE_ATTACHED)
filter.addAction(UsbManager.ACTION_USB_DEVICE_DETACHED)
registerReceiver(usbPermissionReceiver, filter)
eventBus.register(this)
scanUsb()
}
fun scanUsb() {
if (usbConnectedDevice != null) {
disconnectDevice()
}
usbManager.deviceList.values.forEach {
if (it.vendorId == 0xC4F4 && (it.productId == 0x2F37 || it.productId == 0x3F37)) {
usbManager.requestPermission(it, permissionIntent)
return@forEach
}
}
}`requestPermission` will either trigger OS UI to ask the user for permission, or instantly trigger allow/reject based on the previous settings. Since we created IntentFilter for required broadcast events and our PendingIntent for permission check, so once either the device cable gets connected or the user allows the connection, our `usbPermissionReceiver` will be called:
private val usbPermissionReceiver = object : BroadcastReceiver() {
override fun onReceive(context: Context?, intent: Intent) {
val action = intent.action
if (ACTION_USB_PERMISSION == action) {
connectDevice(intent)
}
if (UsbManager.ACTION_USB_DEVICE_ATTACHED == action) {
connectDevice(intent)
}
if (UsbManager.ACTION_USB_DEVICE_DETACHED == action) {
disconnectDevice(intent)
}
}
}So once the device is attached and permission is provided, the `connectDevice` will be called. One more time: the Service is NOT a separate thread, it is just a grouping of background activity, and it is executed inside of the main thread. That means we can’t run infinity loop here that will pump the events, instead, we should do the initialization required and then run the background thread:
fun connectDevice(intent: Intent) {
val device = intent.getParcelableExtra<UsbDevice>(UsbManager.EXTRA_DEVICE)
if (device != null && intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)) {
usbConnectedDeviceConnection = usbManager.openDevice(device)
if (usbConnectedDeviceConnection == null) {
return
}
usbConnectedDevice = device
for (i in 0..usbConnectedDevice!!.interfaceCount - 1) {
usbConnectedDevice!!.getInterface(i).let { intf ->
for (j in 0..intf.endpointCount - 1) {
intf.getEndpoint(j).also {
if ((it.direction == UsbConstants.USB_DIR_OUT)) {
if (usbSendEndpoint == null) {
usbSendEndpoint = it
usbConnectedDeviceConnection!!.claimInterface(intf, true)
}
} else if ((it.direction == UsbConstants.USB_DIR_IN)) {
if (usbReadEndpoint == null) {
usbReadEndpoint = it
usbConnectedDeviceConnection!!.claimInterface(intf, true)
}
}
}
}
}
}
if (usbReadEndpoint == null || usbSendEndpoint == null) {
disconnectDevice()
return
}
usbReaderThread = USBReaderThread()
usbReaderThread!!.start()
eventBus.post(KatDeviceConnectedEvent(katWalk))
}
}Yes, the connection initialization code is a little bit overcomplicated: since we exactly know the device we working with, we know how many endpoints it has and in which order they are, so we could make the code shorter here.
`disconnectDevice` is rather trivial: set the “please, finish” flag, join the tread and disconnect.
Now, the USB connection thread itself:
- Read a packet (with some small timeout).
- Send the read packet into the packet processing function.
- _ If there was no data — send an error into the packet processing function.
- If the PPF returns some data to send — send it.
- If the PPF returns the “sensors updated” event — forward it to the eventBus.
- Repeat until we can.
That way, the whole “intellect” will be implemented inside of the “driver” for the particular VR Treadmill. For the prototype, the AI is as simple as:
- We’ve got a packet: we have a connection
- We’ve got a timeout or an error: we have something going odd.
For the latter we just count the number of bad packets, if that repeats — we reset the connection (disable then enable the stream).
To process the packets we need to deal a bit with Kotlin’s quirks with unsigned types. There are unsigned types, but there are problems with literals of different sizes and casting one to another works weird. To make things work we should cast everything to UByte, even literals, making odd expressions like `(bytes[2].toUByte() == 0xAAu.toUByte())`. So, incapsulate all these quirks into tiny support functions; then finally sit and comprehend the normalization of Quaternion and viola, the final result:
class DirectionSensor : Sensor() {
protected var _direction: Quaternion = Quaternion.identity()
protected var _angleDeg: Float = 0f
protected var _angleZero: Float = 0f
var direction: Quaternion
get() = _direction
set(value) {
_direction = value
val _angle = atan2(2 * (value.w * value.y - value.x * value.z), (value.w*value.w + value.x*value.x - value.y*value.y - value.z*value.z))
_angleDeg = (_angle * 180.0f / Math.PI).toFloat()
}
val angleDeg: Float
get() = normalAngle(_angleDeg - _angleZero)
fun normalAngle(x: Float): Float {
if (x > 360f) {
return x - 360f
}
if (x < 0f) {
return x + 360f
}
return x
}
override fun parsePacket(packet: ByteArray) {
val m15 = 0.000030517578125f // 2^-15
val q1 = readShort(packet, 7)
val q2 = readShort(packet, 9)
val q3 = readShort(packet, 11)
val q4 = readShort(packet, 13)
direction = Quaternion(
(+ q1 - q2 - q3 + q4) * m15,
(- q1 - q2 + q3 + q4) * m15,
(+ q1 + q2 + q3 + q4) * m15,
(+ q1 - q2 + q3 - q4) * m15
).normalized()
if (packet[25].toInt() < 0) {
_angleZero = _angleDeg
}
}
}
class FootSensor : Sensor() {
protected var _move_x: Float = 0f
val move_x: Float
get() = _move_x
protected var _move_y: Float = 0f
val move_y: Float
get() = _move_y
protected var _shade: Float = 0f
val shade: Float
get() = _shade
protected var _ground: Boolean = false
val ground: Boolean
get() = _ground
override fun parsePacket(packet: ByteArray) {
_move_x = readShort(packet, 21) / 59055.117f
_move_y = readShort(packet, 23) / 59055.117f
_shade = packet[26].toInt() / 127f
_ground = (packet[9] == 0.toByte())
}
}To communicate with USB we should also add a Queue, where we’ll put the packets to send whenever we should (start, stop, turn LED on and off). That’s the final driver.
Time to compile, upload to the phone, get bored with the cable on/off thing and lear how to enable and setup the wifi debug on the phone, reboot the PC, learn how to deal with not working wifi pairing (sometimes even reboot doesn’t help, in this case — run sequence
adb pair $IP:$pairport $code
adb connect $IP:$connportnote that pairing port and connection port are different, you can find both in the wifi debugging settings on the phone if necessary), finally, we can just keep treadmill cable connected and update our app whenever we want and even control the phone from the studio UI! Neat!
Great, test on the phone, happy, install and try on the headset — and, magic! it works!
Time to share the result with the indie game developer I’ve mentioned above, the one who has plans to embed the native gateway into his game. He’s excited and made a video with massive plans to share within a KAT community on Discord.
… He also gave me useful feedback and a bugs list that I have in this version: sometimes after the connection is open for a while, if you let the treadmill or app or headset get to sleep, it doesn’t always wakes up, you should restart the app/quest/etc. Definitely, I should read the documentation on Services, Threads, background processes… Also, I am happy to take the fix as a pull request!
In the next episode of the saga
So, as we have massive plans — we have to bring them to life! To do so we should first cut the wire — nobody wants to play a standalone game on a treadmill with the cable. Do we need the cable at all? Let’s try to fix the sensors’ firmware, expand their functionality, and, of course, let’s go even deeper!
Links
- KatWalk C2 Saga, EP1: “playing actually” on [Habr], [Medium] and [LinkedIn
- KatWalk C2 Saga, EP2: “to speak alien” on [Habr], [Medium] and [LinkedIn].
- KatWalk C2 Saga, EP3: “cutting the wire” on [Habr], [Medium] and [LinkedIn].
- KatWalk C2 Saga, EP4: “playing with firmware” on [Habr], [Medium] and [LinkedIn].
- KatWalk C2 Saga, EP5: “overclocking and bugfixing” on [Habr], [Medium] and [LinkedIn].
