V24 Siglos Development Update - final phase of testing

We’re excited to announce that formal testing has begun on V24 - otherwise known as, Siglos. With this release we’re addressing areas of instability within the node, as well as refining several subsections that have evolved over the years. 

We’ve outlined some of the most significant improvements below. But before we jump in, just a quick note to all our node operators:

Thank you for sharing our effort and enthusiasm in continually working to improve nano. Please remember, release candidate versions are subject to rapid change, so representative nodes shouldn’t update until the official release. We’ll let you know exactly when testing is complete and it’s time to upgrade. 

Until then, let’s look in more detail at some of the improvements in store for V24.

Unit testing stability:

One significant advancement for V24 is improved unit test stability and clarity. For a long time, we’ve had a discipline of including a unit test for each functional change added to the node. This discipline allows us to make more significant changes to the node while ensuring there aren’t regressions. Unfortunately over time, older unit tests were starting to fail intermittently, which was slowing down development progress.  

We took several weeks reviewing, documenting, and improving the node’s unit tests which has significantly aided subsequent development work.

Ascending bootstrap:

V24 brings three major changes designed to improve bootstrap reliability:

1. We’ve added the ability for the bootstrap process to request blocks in ascending order. Doing this allows blocks to be inserted in their natural order, greatly decreasing reliance on the unchecked table.
   
   
2. We’ve created a new set of stateless bootstrap messages which allows bootstrap query/response to be sent through the node’s real-time socket. This also means the node doesn’t need to open secondary sockets in order to perform bootstrapping. Not needing to open additional sockets also allows nodes behind strict inbound firewalls to participate in bootstrapping.
   
   
3. We’re working on a new bootstrapping algorithm that is statistical and more randomized, making use of the above improvements. This algorithm will run continuously in the background, adjusting speed depending on sync status and will eventually supersede the legacy bootstrap algorithm.

Rebalancing buckets:

In V23, we added the election scheduler process and set out an initial schedule of “buckets” in which transactions get prioritised for confirmation. This initial schedule uses powers of 2 from 0 to 128 to specify the lower bound for each bucket. This was a simple way to initially balance buckets but it needed refining to operate in the range of natural transactions. We’ve made an iterative improvement in this area, allowing our limited development resources to work on other advancements. 

In addition to rebalancing the scheduler buckets, we also adjusted the scheduling algorithm to consider both previous and subsequent balances. This addresses an issue with the scheduler where sending the full balance of an account would put the transaction in the lowest priority tier.

Hinted elections redesign:

During heavy spam attack, it’s possible that different nodes will observe the same blocks arriving at different times (for example, due to slow/fast hardware, network connection). This could negatively impact synchronisation of the election scheduler. Hinted elections is a mechanism that helps the standard scheduler get back on track.

A parallel election scheduling algorithm caches and looks at votes received from the network and starts elections for blocks that received the most voting weight and weren’t already prioritised by the standard election scheduler. The number of elections started via hinting is limited (by default 1000 vs 5000 for normal elections) and shouldn’t have an impact on normal elections, but helps the network stay in and get back to synchronised state faster.

Removal of UDP code:

The nano protocol was originally written using UDP and later transitioned to TCP. The UDP server has been disabled for several versions however we still needed to maintain the UDP code because of its use in unit tests. Most of the unit tests have been rewritten and most of the UDP code been removed.

What we've observed so far in testing...

We’ve developed testing tools that let us measure and improve the node faster than ever before and we’re already seeing significant performance improvements in Siglos when compared to V23.

Our primary focus for V24 has always been on improving node reliability, which we’ve achieved. The performance gains we’ve seen are a natural, positive consequence of this improved reliability. 

We are really looking forward to formally releasing Siglos - once testing is complete - and of course, we’ll continue to improve performance and reliability in V25 and beyond on our road to commercial grade. We want to thank everyone in our community and ecosystem for helping to make this happen. 

Watch this space for further updates! 

If you would like to help out by making C++ contributions to testing V24, check out our handy guide on exactly how to do this.