XELIS Hardfork/Network Upgrade 1.16.0 Post-Mortem Review
On December 30, 2024, the Xelis network underwent a significant update, necessitating a hard fork upgrade to version 1.16.0. This upgrade introduced critical improvements to enhance security, scalability, and usability. The hardfork event was triggered by various factors, including the need to address a bug brought forth via our robust bug bounty program, planned optimizations for performance, and planned introduction of new features.
Key Enhancements in Version 1.16.0
1.) zK Proof Improvement (Bug Bounty Identified)
- Improve verification by ensuring the handle of each source commitment is correctly built as we do for the receiver, preventing someone to brick its account or do any additional damage.
2.) ZK Proof Transcript Improvements
- Enhanced Zero-Knowledge Proofs by including source ciphertext in the transcript. This makes ZK Proofs directly linked to a balance version, instead of being only indirectly linked through the commitment equality proof
- Improved Commitment Equality Proof verification to add a check that the randomness of the sender is correctly built for the sender public key, just as we do for the receiver
3.) Fast Sync P2P Protocol Rework
- Addressed an issue where fast-synced nodes could get stuck when mining due to incorrect transaction invalidation.
4.) Multisig Wallet Support
- Introduction of multisig functionality, enabling users and exchanges to secure funds collaboratively.
- This feature is critical for institutional adoption and fund security.
5.) Storage Serialization Order Rework
- The storage serialization order for versioned data was reworked to streamline the codebase, reducing redundancy and improving maintainability by implementing a generic method across multiple files.
6.) RPC API Changes
- Several refinements and additions to the RPC API to improve usability and extend functionality for developers.
7.) Updated P2P Key Exchange
- Each daemon now generates a unique Diffie-Hellman (DH) keypair by default to prevent tracking nodes based on public keys.
- Enhanced peer communication security by encrypting key exchanges with DH-derived shared secrets.
- Added options for handling mismatched peer keys (ignore, reject, warn).
8.) Commit Point on Chain Sync
- Introduced a commit point mechanism to allow safe reorganization by preserving the original chain in case of failed reorg attempts.
- This enhances resilience against malicious reorg attempts.
9.) Smart Contracts Testnet Release
- The first testnet version of Xelis Smart Contracts with XELIS-VM (Silex language) support was launched.
- Enables developers to experiment with and deploy decentralized applications.
10.) Storage Optimizations Fixes
- Several improvements were made to the storage system
- Optimized conflict resolution, preventing performance degradation with larger chains.
11.) New Block Version Implementation
- Updated transaction selection rules to prevent duplicate inclusion in the same DAG branch.
- This resolves potential chain split vulnerabilities.
12.) Balance and Ownership Proofs
- Introduced new cryptographic proofs to enhance transparency & regulatory compliance where users can expose their balances or prove they own a certain number of coins without exposing their private keys or wallets.
- Once audited, these proofs will facilitate direct verification on the blockchain explorer.
Challenges Faced During Hard Fork
Issue: Mining pools & user nodes continued operating on the pre-fork chain, resulting in temporary chain splits.
Resolution Process: The Core XELIS development team collaborated directly with each mining pool to ensure they had updated their chains and were correctly synchronized with the appropriate nodes. The hard fork was successfully restarted within an hour once all pools were updated. In addition, improvements were made in rewind chain process by optimizing it to be scalable and unlinked to chain size.
Root Cause: During local tests and private testnet, the hardfork went through our tests flawlessly. During the actual rollout, difficulty was much higher on mainnet and this was not able to be reproduced on testnet. The end result of this was the creation of a lot of blocks simultaneously which froze several nodes due to continuous reorg proposals from other nodes.
Additional issues identified was the miscommunication on part of XELIS team and the short timeframe between announcing the required update and its execution. Additionally, insufficient collaboration with pool operators and the timing of the update — occurring during the holiday season when many operators were unavailable — further complicated the process.
Future Improvements: To prevent similar issues, any future mandatory network upgrades will be communicated well in advance with agreed upon dates. This will provide ample time for necessary updates and thorough testing by third-party mining pools. In addition, in the future, on each hard fork block transition, a fixed difficulty will now be set.
Goodwill Compensation: To support miners and mining pools during these challenges, the XELIS team provided financial compensation from the development fund. This compensation, aimed at addressing any inconvenience and downtime experienced, is a rare occurrence in the industry and underscores the XELIS team’s commitment to delivering a miner-friendly and user-centric protocol.
Lessons Learned
1.) Comprehensive Pre-Deployment Testing
- Increased test coverage across third-party entities, not just internal testing.
2.) Enhanced Communication with Stakeholders
- An extended transition period and improved notifications for miners, exchanges, and node operators to facilitate compliance and smooth adaptation.
Conclusion
Despite the challenges faced during the Xelis hard fork, the team was able to address issues and restore network stability very quickly with no long-term impact to users. Moving forward, continuous improvements in processes and communication will help ensure smoother transitions in future protocol upgrades.
Following a thorough post-mortem analysis conducted in collaboration with core developers and third-party stakeholders, no evidence has been found indicating that any of the identified bugs have been exploited prior to updates.
Future Plans and Upcoming Updates
Looking forward, the Xelis development team plans to further refine network performance, introduce additional features on Mainnet, and conduct audits to ensure long-term success. Upcoming updates include:
- Smart Contract Launch on Mainnet
- Optimization of Consensus Algorithm
- Bolstering the Bug Bounty Program to include formal Third-Party Audits to further build user confidence
Conclusion
The successful deployment of Xelis 1.16.0 marks a significant milestone in the network’s evolution. Despite minor initial challenges associated with the hard fork, the implemented changes have strengthened the network’s integrity, security, and overall functionality. The lessons learned from this experience will continue to guide the team as they work towards mass adoption of XELIS