Exploiting the Lightning Bug was The Ethical Choice

Shinobi is an opinion editorialist in the Bitcoin space. He is a self-taught educator and tech-oriented Bitcoin podcast host. Burak was again the developer responsible for triggering this vulnerability. This time, it was clearly deliberate. It was again an issue with code that parses Bitcoin transactions above the consensus layer. As I mentioned in my article on the previous bug that Burak triggered: Before Taproot, there were limits on the size of the script and witness data that could be included in a transaction. These limits were removed with the activation Taproot. Only the block size limit was left to limit the transactions. The problem with the last bug is that, despite the fact that the consensus codes in btcd were properly updated to reflect this change the code handling peer–to-peer transmission (including parsing data before receiving or sending) did not properly upgrade. The code processing blocks and transactions that preceded it failed to validate for consensus, and the block in question never got validated. This was a very similar situation. Another limitation in the peer-to–peer section was the incorrectly enforcing a restriction of witness data, limiting it to a maximum 1/8 of block size instead of the full block size. Burak created a transaction with witness information that was just one weight unit higher than the strict limit. This caused btcd to stall again at that block height. This transaction was non-standard, meaning that it was valid under consensus rules but not valid under default mempool policy. Nodes will not transmit it across the network. Although it is possible to have it mined into blocks, the only way to do this is to give it directly to a miner. This is what Burak did using F2Pool. This is a great example of how any code that is intended to validate and parse Bitcoin data must be thoroughly audited to ensure it conforms to the requirements of Bitcoin Core. It doesn’t matter whether that code is the consensus engine to implement a node or just a piece of code that passes transactions around for a Lightning Node. This bug was actually right above the one last month in our codebase. Lightning Labs didn’t even discover it. AJ Towns reported the bug on October 11, two days after Burak’s 998-of-9999 multisig transaction. It was posted publicly on Github for 10 hour before being deleted. The fix was made but not released. It was intended to be quietly fixed in the next LND release. This is a fairly standard procedure for a serious vulnerability. This is especially true for projects like Bitcoin Core, where such a vulnerability could cause serious damage to its base-layer network/protocol. Burak demonstrated that this particular bug presented a significant risk to LND users funds. It was also located right next to a previous bug with the same risks. This raises the question of whether quiet patching is the best approach to dealing with vulnerabilities such as this, which can expose users to theft of funds. Because their node is unable to detect old channels and penalize them properly, it is not possible to determine if the quiet-patch approach is the best. As I explained in my article on the last bug: If a malicious actor had discovered the bugs before a well intentioned developer, they could have opened new channels to vulnerable nodes, routed all the contents back to themselves, and then exploited them. They would then have the funds under their control and be able to close the channel in the initial state, effectively doubling their money. Ironically, Burak actually helped LND users by actively exploiting the issue in a way that was ironic. Users were now open to attacks from their pre-existing peers, with whom they had open channels. However, they could no longer be targeted with new channels. Their nodes became stalled and could not recognize or process payments from channels that were opened after the block that had stalled their node. While it didn’t eliminate the possibility of users being exploited completely, it did reduce that risk for people with whom they had a channel. Burak’s actions mitigated it. This type of response to the bug was sensible to me. It minimized the damage, made users aware about the risk, and allowed it to be quickly fixed. LND wasn’t the only thing that was affected. Liquid’s pegging system was also affected, and needed to be fixed by the federation’s functionaries. Older versions Rust Bitcoin were also affected, which caused the stall effect to some block explorers (an implementation of Electrum Wallet’s backend server) and electrs instances. With the exception of Liquid’s peg eventually exposing funds funds to the emergency recovery key held by Blockstream following a timelock expiry — although, realistically, everyone knows who to pursue in the heist-style movie plot that Blockstream stole these funds — these other issues never placed anyone’s funds at any risk. Rust Bitcoin had actually fixed this bug in newer versions. This apparently did not result in any communication with other codebase maintainers to highlight the possibility for such issues. Only the active exploit of the bug on the network made it known that the issue existed in multiple codebases. This raises serious issues regarding vulnerabilities in Layer 2 software on Bitcoin. The first is the seriousness of the codebase audits for security bugs. Second, how the integration of new features is prioritized. It is quite telling that security is not always prioritized, as this second bug was not found by the codebase maintainers even though it was right next to the first bug. No internal audit was done on that codebase after one major bug that exposed users’ funds. It took someone outside the project to find it. This does not show a priority to protect users’ funds rather than building new features to attract more users. This is second because Rust Bitcoin already addressed the issue. This shows that there is a lack in communication between maintainers of different codebases regarding bugs like this. This is understandable as there are different codebases. However, it doesn’t mean that someone who finds a bug in one codebase should immediately think, “I should notify other teams writing similar software using totally different programming languages.” It’s not like you find a bug on Windows and then decide to report it to the Linux kernel maintainers. Bitcoin as a protocol to distribute consensus across a global network of computers is quite different. Bitcoin developers might consider these thoughts when it comes to vulnerabilities in Bitcoin-related software. This is especially true when it comes to understanding and parsing consensus-related data. Finally, protocols like Lightning that depend on the blockchain being observable at all times in order to be able react to old channel state to maintain security should be kept independent of verification and parsing. If not, it should be delegated to Bitcoin Core, or data directly derived. Core Lightning is designed in this manner, connecting to a Bitcoin Core instance and relying entirely on it for the validation of transactions and blocks. LND would have been able to work in the same way if these bugs in Bitcoin Core had not affected LND users in a manner that could have put their funds at risk. Whatever way things are handled, whether outsourcing validation completely or minimizing internal validation and approaching this issue with more care, this incident highlights the need to change how Layer 2 software interacts with consensus-related data. Everyone is very fortunate that this incident was not exploited maliciously by an actor. Instead, it was a developer proving a point. Shinobi contributed this guest post. These opinions are not necessarily those of BTC Inc. or Bitcoin Magazine.


Add a Comment

Your email address will not be published. Required fields are marked *