By Darshit Suratwala (Blockchain Engineer) and Jay Thakrar (Product and Strategy) at Scale3
TL;DR:
Here is a brief overview of what day-to-day blockchain node / validator deployment and management looks like at Scale3:
- Engineers provision multi-tenant multi-geo nodes / validators using Scale3’s Nodepilot
- Once set up, nodes / validators are automatically onboarded to Autopilot for end-to-end observability. Autopilot’s observability platform includes monitoring, alerting, and logging.
- Alerting is instrumental in remaining proactive with managing nodes / validators. Using Autopilot, engineers configure the type of host-specific and validator-specific alerts and where to recieve the alerts, such as PagerDuty, Slack, Discord, Telegram, and e-mail. Additionally, engineers add other team members to the account to ensure all team members are notified of any alerts.
- When an upgrade alert is sent, an engineer acknowledges the alert message for a new node upgrade, and then executes the node upgrade via Nodepilot. Using Scale3’s pre-built binaries, Nodepilot upgrades the node to the latest version available.
Managing Nodes at Scale3:
At Scale3, we manage blockchain nodes to enhance our products by providing network intelligence, monitoring dashboards, intelligent logging, on-chain alerts, and other valuable functionalities.
Keeping up with blockchain node upgrades is a significant operational overhead. To address this challenge, we developed a simple-to-use tool called Nodepilot, which helps us run and maintain blockchain nodes / validators on GCP, AWS, Azure, and on-prem data centers with minimal operational effort. With Nodepilot, anyone can spin up a blockchain node within a matter of minutes. Let’s walk through how we utilize our own platform to keep our Self-Hosted blockchain nodes up to date.
Setting up a Blockchain Node using Nodepilot
Nodepilot simplifies the process of setting up a blockchain node, regardless of whether you choose GCP, AWS, Azure, or any other data center.
Nodepilot simplifies the process of provisioning a node in any cloud (GCP, AWS, or Azure) environment. To get started with Nodepilot, complete the following steps:
- Sign-in to your cloud provider via Nodepilot
- Select blockchain network (Ethereum, Sui, etc.)
- Select blockchain network environment (Testnet, Mainnet, etc.)
- Select cloud provider configurations (VPC, region, etc.)
- Node type (full node, archival node, etc.)
After entering the above inputs, Nodepilot proceeds with configuring a node / validator. It’s equipped with knowledge of the ideal server requirements, including CPU, memory, disk specifications, and the essential open ports tailored to the selected blockchain network. Within minutes, Nodepilot provisions a new blockchain node in your own cloud infrastructure.
Following the deployment of a new node, we will need to establish monitoring, alerting, and observability infrastructure through Autopilot:
- Using Nodepilot, the new node is automatically onboarded to Scale3's Autopilot for monitoring, logging, and alerting.
This helps us with uptime preservation, swift issue detection and response, trend analysis, and cost-effective operations. - Since Autolog is enabled, we can then stream, store and search logs from our nodes live, reducing log rotation and storage issues.
- Lastly, the setup of node update alerts is used when a new version of node software is released, for immediate alerts on all alerting platforms.
Keeping up with node updates
To ensure seamless updates, we developed an in-house system called Hephaestus, responsible for automatically building node software binaries for every new release.
Furthermore, we implemented blockchain release alerts on all alerting platforms including PagerDuty, Slack, E-mail, and Scale3’s community Discord. These alerts promptly inform engineers about the latest blockchain releases, keeping them well-informed and up-to-date.
When an alert is received, engineers can respond immediately by making an API call to Nodepilot. In response, Nodepilot utilizes the binaries crafted by Hephaestus to efficiently update the blockchain node.
By integrating Hephaestus, alerts, and Nodepilot, we built a well-coordinated system that streamlines the node update process, reducing manual efforts and optimizing the overall efficiency of our blockchain infrastructure.
Nodepilot vs. RPC Providers
The following considerations below may help you decide whether to leverage an RPC provider or run self-hosted nodes.
TLDR;
- Use a blockchain RPC provider when you need a quick and easy way to interact with a blockchain, have limited resources, or don’t require full control over your data.
- Use a self-hosted blockchain node when you need greater customization, data privacy, and control, and anticipate heavy usage in the long term.
Considerations:
Resource Requirements
- Blockchain RPC Providers: If your application does not require constant access to a blockchain or doesn’t perform resource-intensive tasks, an RPC provider might be sufficient. It offloads the node’s resource burden to the provider.
- Self-Hosted Node: Resource requirements for a self-hosted node can be significant, especially for popular blockchains like Solana or Ethereum. You need sufficient hardware resources, storage, and bandwidth to handle blockchain data.
Data Privacy and Security
- Blockchain RPC Providers: Using a third-party provider means trusting them with your data, which might not be suitable for applications that require a high level of privacy or security.
- Self-Hosted Node: Hosting your own node allows you to have full control over your data and transactions, enhancing privacy and security.
Customization and Flexibility
- Blockchain RPC Providers: usually offer standard APIs, limiting the level of customization you can achieve. This might be sufficient for many use cases but can be restrictive for more complex applications.
- Self-Hosted Node: Running your own node grants you complete control and allows you to customize your interactions with the blockchain as needed.
Cost Considerations
- Blockchain RPC Providers: Some providers might offer free or low-cost access for limited usage, but you might incur significant costs when expanding operations as your usage scales up.
- Self-Hosted Node: Running your node might involve higher upfront costs, but it is more cost-effective for heavy usage and growth over the long term.
Reliability and Redundancy
- Blockchain RPC Providers: Reputable providers often offer high uptime and redundancy, ensuring your application remains operational.
- Self-Hosted Node: The reliability of your self-hosted node depends on your setup and maintenance efforts. Downtime or connectivity issues could affect your application, but can be mitigated with the help of Scale3’s tools.
Conclusion
The decision to utilize self-hosted nodes or RPC providers largely depends on your specific use case along with the several major factors outlined above.
Self-hosted nodes offer greater control over data privacy, minimal latency, more flexibility in how you interact with the blockchain, and can be more cost-efficient while not sacrificing on your current security posture. However, self-hosted nodes require diligent maintenance to ensure high reliability and availability, which can be streamlined via the Scale3 platform.
Conversely, blockchain RPC providers may be easier to get started with in terms of interacting with the blockchain, however, they may also pose challenges regarding data privacy, latency / performance, flexibility in how you interact with the blockchain, and may be more expensive depending on usage.
How to get started?
Our goal with Nodepilot is not to be a node provider but to be a node enabler. We believe in empowering our users with complete control over their blockchain servers. With Scale3’s observability platform as their trusted companion, users can confidently take charge of their nodes’ reliability with ease. To get started with Nodepilot, click here.
We want to hear from you.
Everything we do at Scale3 is in service of making your experience as builders as easy as possible. If there’s something you want to see in terms of Nodepilot moving forward or if you have feedback on the new experience, please reach out to us!
