Is a software company working on financial solutions built on a blockchain, such as decentralized asset exchanges, banking, project funding
On Monday, July 10th, a software company BitShares faced an incident occurred on the BitShares network that caused an unplanned interruption of block production.
As the company reports in the official announcement, all block producers have been affected by a memory corruption that was caused by an automatic resize of a
flat_index container that resulted in an unrecoverable stale state. This has happened for the first time in over two years of blockchain operations.
Distributed database that is used to maintain a continuously growing list of records, called blocks
Luckily, after several core developers debugging the code, the cause was identified and a patch was quickly delivered to the block producers. Shortly after that, the blockchain recovered and new blocks have been generated. All transaction that made it into the blockchain prior to this incident were unaltered.
The team argues that it is absolutely necessary that everyone is aware that the nature of the patch requires all nodes to apply the patch, accordingly, in order to sync back with the blockchain. The exchanges and third party providers are to update their back-end to tag
2.0.170710 and rebuild. The following steps facilitate this update:
git fetch git checkout 2.0.170710 git submodule update --init --recursive make
As it turned out after further investigation, the root cause was an inappropriate implementation of
flat_index::remove that only zeroed an entry instead of removing the whole element from the container, BitShares team detailed. What happens is that a new bitasset has been proposed but the proposal wasn’t fully approved. The bitasset data was created in the database and then zeroed instead of being removed after expiration of the proposal. This is the first time in the history of the chain when
flat_index::remove was called.
The conclusion is that
flat_index is not made for having items removed. Proposals can always lead to items being removed, which means that
flat_index is not suitable for the job and should be replaced in all places.