At least half of the nodes for the Cardano network briefly went offline over the weekend, there was a brief period of degradation.
According to Input Output Global, an anomaly caused 50% of Cardano nodes to disconnect and restart. The outage impacted relay nodes and block-producing nodes, but edge nodes seemed to have been unaffected. IOG believes the outage was triggered by a transient anomaly that caused one of two reactions in the node. As such, some disconnected from a peer, while others threw an exception and restarted.
IOG said the transient issues were considered in the design of the Cardano-node and consensus. The Cardano network recovered quickly without external intervention. Cardano’s systems behaved exactly as expected. The anomaly occurred between blocks 8300569 and 8300570. Block production continued but was slow. IOG said the impact was low, similar to the delays that occur during normal operations. And most nodes recovered automatically, depending on the SPO of choice.
Cardano is looking into the root cause of the anomalous behavior. The network is implementing further logging measures in addition to its regular monitoring procedures. Tom Stokes, the co-founder of Node Shark and a Cardano SPO, said more than half of the listed nodes were affected. He highlighted a chart that showed where the network sync dropped from 100% to slightly above 40% for over 300 reporting nodes. Stokes said the network sync recovered to around 87% after the drop. However, the network did not immediately return to its previous level of 100%.
