Zigbee2mqtt: How to add support for a new Tuya-based device (part 2)
Important note: The hexadecimal values in this article are prefixed with
0x
.
In the previous article, I explained how I ended with 4 Zigbee radiator valves that required a Tuya gateway to control them remotely, which initially I didn’t have.
Once the Tuya gateway arrived, I went down to work. First, I install the gateway, the Smart Life app (by Tuya), and then pair one TRV with the gateway.
The catalog available on the app was confusing. Some devices like the gateway itself appeared more than once with the same label. In the case of the TRV, I had to choose a generic Zigbee TRV device from the list, which, lucky me, seems to be the correct one. After successfully paired the TRV, the new device appeared on my dashboard.
The experience sadly got worse. The UI created by Siterwell looks ugly, and the feedback for user actions took several seconds on some (apparently random) cases. I was not worried about it. I was not planning to use that UI anyway. The plan was to control all of them from Home Assistant.
Due to curiosity, I tried the integration of my newly created Smart Life account (Tuya) with Home Assistant. That integration exists and had documentation on the Home Assistant website but, in practice, it didn’t work with my Siterwell TRVs. The temperature values were not correctly formated (as you will see ahead, we need to move one decimal position to the left to all temperature values returned by the TRVs), and the change of modes and target temperatures were not working at all. That reinforced my opinion that the only viable option for me was to make them work with zigbee2mqtt.
Now that I have the Tuya gateway (the Zigbee Coordinator) and the Radiator Valve (Zigbee End Device), it was time to start the next part of my plan: spy their communication.
I only had one CC2531 USB adapter. The same one zigbee2mqtt was using to act as a Coordinator. I had to borrow it and re-flash it with another firmware made specifically to sniff Zigbee communication. The sniffer_fw_cc2531.hex
file provided by Texas Instruments. I found the instructions to achieve that on the zigbee2mqtt documentation.
I didn’t want to spend many on buying a CC Debugger from Texas Instruments only to use it once. So I decided to follow the instructions that require the use of a Raspberry Pi’s GPIO pins.
Flashing with a Raspberry Pi
Before start, I strongly recommend you to use a programming download cable for the CC2531/CC2540. I used my soldering iron and, things went ok at the beginning, but it didn’t end well.
Please follow the instructions carefully. Use this map to connect the right pins of your CC2531 with the GPIO pins of your Pi and then run ./cc_chipid
when you’re done.
The instructions said cc_chipid should print ID = b524
on the screen. Printing 0000
or ffff
could mean something is wrong with the wiring. But guess what. There’s another explanation.
Here is a good tip for you: If you keep getting 0000
after checking your wiring, then try these possible solutions:
- Increase the time delay by using the -m option.
$ ./cc_chipid -m 300
- Fix the CPU speed of your pi before running the command.
sudo bash
$ performance >/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
exit
Sniffing Zigbee traffic
Once you have your cc2531 USB stick flashed with the sniffer firmware, its time to go back to this tutorial.
I used the ZBOSS tool for Windows. The first thing this tool asked me was to select the Zigbee channel I wanted to spy. Zigbee channels go from 11 to 26. I had to test one by one till I see Wireshark print something. My Tuya gateway was using channel 24. It could be another in your case.
The Zigbee join process
When a Zigbee End Device (ZED) joins a network, the Zigbee Coordinator (ZC) sends to the ZED the encryption key of its Zigbee network.
This network key travels in an encrypted frame. The encryption key of that encrypted frame is called the Trust Center link key, and its value is publicly known. This value is the same for most (or maybe all) devices, and it is hardcoded on the software used the ZEDs (Zigbee End Devices). The value of this key is 5A:69:67:42:65:65:41:6C:6C:69:61:6E:63:65:30:39
and you have to register it on Wireshark to be able to decrypt the frames between the ZC (Zigbee Coordinator) and the ZEDs during the joining network process.
Zigbee its a cool technology but has been not exempt from problems. One, in particular, has to do with how secure the process is to join new devices to your network. During that process, your secret network key travels through the air in a frame encrypted in with a publicly known key. Anyone with a basic knowledged of the Zigbee protocol could intercept that key if is close enought.
This unique network key is what you need to intercept to be able to decrypt the messages between the Tuya Gateway (ZC) and the ZED devices. This key will appear on Wireshark labeled as the Transport key. Extracting it and register it next to the Trust Center link key should be fairly simple.
Once Wireshark has the network key, it will automatically decrypt the payload of all the encrypted packages.
The payload structure used by Tuya Platform
In the case of the radiator valve Siterwell GS361, my Zigbee device, soon after the ZED receives the network key, it started to use the Zigbee cluster code 0xef00
. This is the cluster code that the Tuya platform uses to communicate with its devices.
I identified the use of three commands under this cluster:
- 0x00 Used by the ZC to send commands to the ZEDs.
- 0x01 Used by the ZED to inform of changes in its state.
- 0x02 Send by the ZED after receiving a
0x00
command. Its data payload uses the same format as the0x01
commands.
On the Zigbee Cluster Library specification (ZCL), the Zigbee Alliance defines the structure of the Application Layer. The AL can be divided into Header and Payload. The structure of the Header is described on the specification, but the structure of the payload it’s decided by each manufacturer.
This is the payload structure used by Tuya (as far I can tell).
The green side (right) of the table is the payload structure used by (maybe) all the devices that use the Tuya Platform.
- status: In all the packages I captured, its value was always zero.
- transid: Seems to have the same purpose as the Transaction Sequence Number. Its purpose is to relate two or more frames as participants of the same sequence. e.g., A command
0x02
emitted by the ZED in response to a command0x00
received from the ZC will have the sametransid
number. - dp: Identifier of the action/type of message. e.g., The ZED uses the
dp
value0x0302
to notify the room temperature to the ZC, and0x0404
when the heating mode changes. - fn: It had the value 0 in all my capture frames.
- data: Siterwell GS361 devices send the value formatted as an
octetStr
(code0x41
according to the Zigbee Cluster Library).octetStr
values have a variable length. The first byte contains the length of the value.
Notice that the “dp” code uses the Big Endian format. That means that its value has to be read from left to right. For example: for a hex value
03 02
as “dp” it has to be understood as02 03
. This value converted to decimal results in515
, the code used by the this Siterwell device to inform the current temperature of the room.
Identifying the commands
With this knowledge identifying the command codes for each action requested by the ZC or ZED should be a matter of repeat this pattern as long as needed.
- start listening frames with Wireshark
- executed the action (e.g., increase the temperature by one degree)
- stop capturing frames
- identify the frame that carried the order
- identify the frame’s response (if there’s is one)
- note the values of the header fields and the payload
When I did it for my device, I ended with a table like this.
This table has useless data thought. Fields like the transaction sequence number
, status
, transid
, and fn
could be ignored. The first and third because they are generated sequentially and will be different on each request and the rest because they were always zero.
Wireshark filters
- 0x00 ZC -> ZED:
(zbee_aps.cluster == 0xef00) && (zbee_zcl.cs.cmd.id == 0x00)
- 0x01 ZED -> ZC:
(zbee_aps.cluster == 0xef00) && (zbee_zcl.cs.cmd.id == 0x01)
- 0x02 ZED -> ZC:
(zbee_aps.cluster == 0xef00) && (zbee_zcl.cs.cmd.id == 0x02)
Remember: The only different between 0x01 and 0x02 commands is that 0x02 commands are send by the ZED in response to a recieved 0x00 command sent by the ZC.
Examples
Let’s figure out this frame:
Let’s point the obvious thing first: this is a frame sent by the device 0x0bb4
(the ZED) to the device 0x0000
, the address took by the ZC. The cluster code is 0xef00
(a code chosen by Tuya), and the profile code is 0x0104
, which according to the ZCL specs, is the profile for Home Automation devices.
The Zigbee Coordination (ZC) always takes the address
0x0000
.
The hexadecimal sequence highlighted in blue is the Tuya payload. Let’s decipher it.
First, we copy the values over a table with the structure presented already in this article. Then, reverse the order of the bytes for the field dp because this value uses the Big Endian format.
Lastly, remove the first byte of the field data because its purpose is to declare the total length (in bytes) of its value.
Work In Progress: In the next part, I will show you how to add support for this device in zigbee2mqtt.
Final thoughts
If you have recommendations on how to improve this article, do not doubt to contact me and let me know. I will appreciate it.