Decrypting Project1999 Spawns
Unwrapping the EverQuest network protocol
- Installing ShowEQ For Project1999 in 2020
- Getting traffic to your ShowEQ system
- Understanding Project1999 Protections
- Decrypting Project1999 Spawns (you are here)
- Project1999 Green and /List information
If you made it this far, you have should have compiled version of ShowEQ on a modern Linux distribution, a method of getting traffic to your ShowEQ system, and a general understanding of the protections implemented by the Project1999 developers on the code base. If not, please take a moment and read the previous three posts.
The EverQuest Network Protocol
EverQuest was designed with the networking requirements of a largely pre-broadband Internet. As such, design decisions seem to have been focused on making a very low latency protocol for real-time communication. This was obviously necessary when you had hundreds or thousands of clients interacting in a shared world environment, with an expected 25kb/s download speed.
Here is a summary network communication design decisions made by the EverQuest developers (Verant) at the time:
- UDP Communication Protocol
- Custom concept of packet reconstruction, sequences, re-transmissions
- Compression for smaller packets
- Binary byte-wise flags to lower overhead
- Cyclic Redundancy Checksum (CRC) to ensure packets arrive without modification/corruption
The result is a largely compressed binary data protocol. This is why if you try to visually observe the EverQuest protocol in a network traffic analysis tool, like Wireshark, it would look like gibberish.
While trying to understand the protocol ourselves, we looked online for documentation to further our knowledge. Outside of a handful of posts, such as this slightly too complicated EverQuest Protocol Layer diagram written by ‘randy-girard’ in 2014, we were mostly out of luck.
Fortunately for us, we already have the best explanation of the EverQuest Network protocol written in the ShowEQ source code itself.
ShowEQ Source Code Analysis - Packet Outer-layer
After spending a couple of long nights parsing through old C++ code, it became apparent how the protocol was constructed. Each packet contains a header that describes the type of packet it is, and what type of message it conveys. A lot of the packets are for setting up communication channels between the client and server and determining which options are applied.
One method that immensely helped with the understanding of the protocol was using ShowEQ’s built-in debug handler for tracking execution. The following code below is the original ShowEQ code with added debug messages to explain program flow. Combing this option with the ability to run ShowEQ against already pre-captured sessions (pcaps) allowed us to rapidly iterate through code branches.
EverQuest’s packet format contains multiple message types, but the ones that have the interesting data are OP_Packet (normal packets), OP_Oversized (a collection of large, chunked packets), and OP_Combined which contains two or more separate OP codes. Reconstructing these messages properly ensures that you can see the actual game-related messages being sent from the server to the client and vice-versa.
EverQuest Application Protocol - Inner-layer
Once the packets are identified, decompressed, reassembled (in the case of oversize packets) and positioned correctly you can begin to read the underlying application specific protocol.
The actual EverQuest Application protocol is similarly an ‘OP code’ based protocol where the first two bytes represent the type of command being sent from the server and the rest of the packet contains the application message.
For example, on Project1999 the opcode 0x0920 or OP_NewZone is sent by the server upon changing zones. This packet from the server explains to the EverQuest client which zone files to load and some other zone-specific information. The information is sent as a C ‘struct’, which is essentially an object that contains multiple data types. The struct contains the zone’s names (short and long), the Zone Experience Multiplier or ZEM, where the ‘succor’ or safe spot is located, and many placeholders for modifications or changes over time.
In EverQuest, there are over 100 different Application OP codes varying from things like updating the location of mobs as they walk around to telling you that it started raining. Each one of these OP codes has a corresponding struct, expecting data in a specific format. These data formats and OP codes changed frequently during updates historically, causing ShowEQ to stop working until it could be corrected.
Out of all of the opcodes, the ones of interest to most players would be ones that cannot be observed directly in the client. Out of those, of particular interest are charProfileStruct and spawnStruct.
charProfileStruct contains information about your currently logged in character, including your guild status and position, the amount of experience points (as a raw number) you have earned, and things like money in the bank.
spawnStruct contains comparatively some of the more flagrant hidden data, such where a mob is located, what the mob is holding in their hands, their race/class/level/diety, and other flags like if they’re invis/afk/lfg or a GM. You receive a spawnStruct for every mob that is currently spawned when zoning in, and another one is sent every time a new mob spawns. You also receive them for player characters in the zone.
Project1999 and spawnStructs
So great, all we need to do is find the opcode for the spawnStructs, re-assemble the packets and decompress the packets properly and then we’re good to go, right? Unfortunately, Project1999 introduced additional protections on the spawnStruct related opcodes that are not implemented in normal EverQuest. It should come as no surprise that ‘dsetup.dll’ (described in detail in the third part of the series) implements some additional protections specific to Project1999.
As you can see, by default the spawn information appears to be obfuscated, encoded, or encrypted as the data does not make sense. Often times just reading the spawn data from the server may make ShowEQ crash because of all of the unexpected input and poor error handling.
So what is actually happening behind the scenes here? Well, the spawnsStructs are being encrypted using a key sent to the client upon server connection. The Project1999 developers added a 10-byte hex key to the OP_SendLoginInfo application code, used for encrypting spawns. This key is first sent to the client, and then used (in ‘dsetup.dll’) for decrypting the spawnStructs.
The relevant section of code for decrypting the spawns is located here:
To bypass this protection, you first need to modify ShowEQ to read the key from the OP_SendLoginInfo code, and then apply a null-preserving, rotating XOR to the spawnStructs. The code change takes about ~30 lines of C code added to ShowEQ:
- Modify the ShowEQ Configuration Op Codes document to expect the additional data.
- Modify the ‘struct’ to contain the decryption key.
- Store the key in memory for the session and update when necessary.
- Modify the spawnStruct handling functions to decrypt the entire packet before further processing.
In order to prevent wide-spread abuse, we are intentionally leaving the code patch as an exercise for the reader.
Once completed. You will have a 100% working ShowEQ on Project1999.
Conclusion and Future Work
At the end of the day, the important part of playing EverQuest is enjoying the game. If this helps you enjoy the game more, then we are glad to document this process. We have been using these modifications for almost a year and it has greatly improved our experience.
In order to fully understand the protocols involved, we have written a re-implementation of ShowEQ in python. In addition to the features provided by ShowEQ it contains modern application features like notifications, spawn window tracking, and other goodies. We have no plans to release this publicly at the moment, but that may change in the future.
Acknowledgements
We would like to thank support the ShowEQ community, especially BlueAdept, a true EverQuest OG, who gave us the confidence to get started on this path. Also shout outs to the original ShowEQ developers for making an amazing tool and, of course, the Project1999 staff for giving us hours of free enjoyment.
See you in Norrath.
