Lessons learnt from the pGALA exploit

Security and transparency are key values that drive the work of the pNetwork team. The pGALA issue has highlighted vulnerable points in how elements surrounding the protocol were handled. While there’s nothing we can do to change what happened, we are committed to learning from these weaknesses and improving for the project to return stronger than ever.

Below are some key lessons learnt and ways in which the project intends to improve.

Comprehensive and real-time monitoring

Real-time monitoring is the continuous monitoring of an IT system — this is a useful tool to automatically inspect whether a system is working as expected and to spot any issues. While real-time monitoring was already in place for the pNetwork protocol, this was unfortunately not sufficient to promptly detect that the pGALA smart contract had been compromised.

In the context of a pTokens smart contract, pNetwork’s monitoring system erroneously monitored the token smart contract itself and its so-called “proxyAdmin” smart contract (a smart contract that has control over the token smart contract) only. However, in the case of pGALA, an additional address was in place that operated as the owner of the proxyAdmin smart contract. This owner was not being actively monitored.

The lack of monitoring on the owner smart contract delayed the detection of the loss of ownership over the pGALA smart contract. In fact, the issue was noticed while performing maintenance work on the pNetwork protocol — should monitoring on this third address be in place, the issue could have been spotted much earlier.

A new comprehensive and real-time monitoring system is in the works and includes the whole protocol codebase as well as new alerts.

Security has always been and continues to be a priority for pNetwork. Other than a monitoring system, other security measures that are already in place include:

• security audits. The pNetwork codebase is periodically audited — the latest review has been performed by a leading security auditing firm and it includes the entire pNetwork protocol as well as the latest cross-chain connections (this took several months of work and at the time of writing, the final report is being finalised).
• bug bounty. A bug bounty programme is available on the leading platform Immunefi for responsibly disclosing any potential vulnerabilities affecting the pNetwork protocol.
• insurance coverage. Insurance coverage is available on Tidal Finance for anyone interested in protecting their funds in case of issues.
• custom alerts on unusual cross-chain activity. These alerts automatically detect and enable prompt intervention in case of unusual activity on the cross-chain bridges.

Key management

The pNetwork protocol implements a progressive decentralization path. While providing a variety of other benefits, decentralisation positively impacts key management as the control over the protocol itself is distributed across a number of parties.

The fact that pNetwork is reaching decentralisation in a progressive manner implies that key management is being gradually improved as well. Specifically, the already ongoing upgrade from v1 to pNetwork v2 introduces a more secure, multisig key management operated via GnosisSafe.

Unfortunately, the vulnerability on the pGALA smart contract was found while attempting to upgrade its cross-chain bridge to pNetwork v2. While these measures wouldn’t have prevented the leak of the private key, they would have prevented the actual act of maliciously changing the above-mentioned owner smart contract from one under pNetwork’s control to one under the attacker’s control.

Education and collaboration with industry peers

Wrapped tokens are cryptocurrencies pegged to the value of another original asset — the original asset is “wrapped” into a digital vault and a newly minted token is created to operate on a host blockchain. Because of their nature, wrapped tokens have a different trust model compared to their original underlying asset. While under normal circumstances there is a 1:1 peg between the wrapped asset and the native one, the peg may not always hold (for example in the case of a hack).

The growth of the DeFi and overall cryptocurrencies industry has generated the rise of a number of wrapping and cross-chain solutions processing Billions of Dollars in cross-chain volumes per week. While being a useful and popular tool for crypto users, unfortunately bridging solutions have also suffered a large number of hacks in the last couple of years. The popularity of such a technical system and its notoriously fragile design led to thinking that project-teams and users leveraging such tools were familiar with the risks involved. Unfortunately, this turned out not to be the case.

While pNetwork powers cross-chain interoperability of mainstream assets and project-tokens alike, once the wrapped version is in circulation it is available to anyone and everyone. Blockchain composability makes it possible for the token to be integrated with third-party protocols (for example, centralized and decentralized exchanges, lending platforms and DeFi protocols) without pNetwork’s permission or even without it being aware.

Even if pNetwork didn’t promote the growth of the pGALA ecosystem and was never part of listing pGALA, GALA or pGALA as GALA on any centralized exchange or DeFi protocol, the pGALA issue highlighted an information asymmetry on the risks involved with wrapped assets. It is the intention of the team to work along with industry peers on educating DeFi enthusiasts and crypto users on the risks involved with wrapped tokens and cross-chain protocols.

To make it straightforward, wBTC presents a different trust model and risk level compared to BTC. This is due to the fact that whenever leveraging an asset within an innovative protocol still in its infancy (as all DeFi protocols can be considered Today, given DeFi itself started to flourish just in the last 2/3 years), additional risks are inherently added. The same reasoning applies to pGALA and GALA tokens, and to every wrapped token.

That said, we continue to believe in the key role cross-chain protocols have for the growth and mainstream adoption of blockchain and cryptocurrencies. For this reason, it is the team’s intention to collaborate with industry peers on finding better ways to continue leveraging such cross-chain systems while also improving security for users.

An example of this could be for centralised exchanges that wish to support deposits/withdrawals for an asset from multiple chains to adopt a “redeem on deposit” procedure. This would mean exchanges could automatically redeem the original underlying asset whenever a wrapped asset is deposited (and credit to their users the underlying asset just then, once securely received) and, similarly, automatically trigger the issuance of the wrapped version whenever the withdrawal of a wrapped asset is requested. This would enable users to continue seamlessly to deposit/withdraw an asset to/from the centralised exchange to/from any supported blockchain. At the same time, it would reduce the risks involved for their users as the exchange itself would only safeguard the original asset. The pNetwork team has already taken action to draft guidelines for exchanges to implement with security and ease the process described above.

Emergency contacts

The pGALA issue highlighted the importance of fast and straightforward communication with third parties in case of emergencies. Whenever a vulnerability on a smart contract is detected or a malicious hack happens, timely intervention is of the essence to either avoid loss of funds or limit damages.

Should the attacker have performed a malicious hack, there would have been a much larger impact as the underlying bridge funds could have been stolen and an unlimited amount of pGALA tokens could have been maliciously minted and sold on both decentralized and centralized exchanges, causing the price of GALA itself to be pushed to zero. Because of the level of risk involved, timely action was considered of essence.

The existence of responsive points of contact has effectively contributed to the safeguarding of users’ funds. For example, as the largest centralised exchange, binance.com’s fast response and prompt intervention to the emergency notification led to any negative impacts being significantly reduced.

Because of the nature of the industry, it is the intention of the team to collaborate with industry peers on creating effective emergency procedures that enable coordinated action among multiple parties. As per our part, we will ask projects we collaborate with to share in advance their emergency contacts to ensure any emergency situations are addressed with clear and timely communications.

Resolution timeline

The first part of the Recovery Plan has been successfully completed and provided eligible parties with a new pGALA token. Unfortunately, the second part to return the recouped BNB tokens had to be delayed. While the team remains committed to returning the funds, we underestimated the time needed for certain legal analyses to be performed and legal requirements to be satisfied. As it turns out, crypto times aren’t legal times.

In the urge to be transparent on the funds recouped, timely address the situation and encounter the expectations of the community for a prompt resolution, a too-optimistic timeline was shared as per the second part of the recovery plan. It is the intention of the team to continue working with the relevant entities for the recouped BNB tokens to be released as soon as possible and to keep the community updated.

Conclusion

By sharing publicly the above lessons learnt and improvements intended, we hope for the community to be reassured by the team’s commitment towards improving the system and avoiding any such issues in the future, for other projects in the industry to be inspired at adopting strong security measures and hopefully to avoid any such issues in the future.