Why does Bitcoin use both a transaction ID and a witness transaction ID?
Why does Bitcoin use both a transaction ID and a witness transaction ID?
Loading saved threads...
Aayush Sakariya · External communityPost link
External question — Bitcoin Stack Exchange
Author: Aayush Sakariya
Original post: https://bitcoin.stackexchange.com/questions/131077
License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
While learning Bitcoin transactions and SegWit, I came across the distinction between the traditional transaction ID (txid) and the witness transaction ID (wtxid).
I understand that the txid is the double-SHA256 hash of the serialized transaction excluding the witness data. andd wtxid commits to the transaction including the witness data. SegWit introdduced wtxid partly to solve transaction malleability issues.
What I am having trouble understanding is why Bitcoin needs both identifiers instead of simply replacing txid with a hash that always commits to the witness data. For example, suppose a SegWit transaction has the same inputs and outputs but different witness data. The txid remains unchanged while the wtxid changes. I would like to understand the design consequences of this distinction...
What specifically would break if Bitcoin used only wtxid everywhere?
Why do transaction inputs continue to refer to previous outputs using txid:vout rather than wtxid:vout?
How does this distinction affect the mempool and transaction relay?
Why does the witness merkle tree in a block use wtxids while the normal merkle tree uses txids?
Is the separation primarily a backwards-compatibility decision, a consequence of how SegWit fixes malleability, or are there deeper protocol reasons?
I am particularly interested in the historical/design reasoning rather than just the definitions of txid and wtxid. References I've been reading so far include BIP141 and the Bitcoin transaction/SegWit sections of Mastering Bitcoin.
Quote
Report
Pieter Wuille · External communityPost link
External answer — Bitcoin Stack Exchange
Author: Pieter Wuille
Original post: https://bitcoin.stackexchange.com/a/131079
License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
This is a great question. The answer depends on which aspect you're talking about.
To start, it is important to highlight what segwit's goal was. Besides being backward compatible (a softfork) and increasing block capacity it is most known for, its primary purpose was
fixing the transaction malleability issue
. Prior to segwit, chains of unconfirmed transactions were impacted by the inherent malleability of Bitcoin transactions. When two or more parties negotiate a set of dependent to-be-broadcast-later transactions (as in common in many layer 2 and off-chain protocols), those transactions and their validity depends on the txids of the outputs they are spending. As signatures and many other aspects of transactions (see
BIP-62
) are malleable, this breaks those dependent transactions as the txid they refer to can change before inclusion in the chain.
This was solved by moving the malleable parts of a transaction to a separate area (the witness), so that txids are not impacted by it.
What specifically would break if Bitcoin used only wtxid everywhere? Why do transaction inputs continue to refer to previous outputs using txid:vout rather than wtxid:vout?
The desire to solve transaction malleability breaking chains of unconfirmed transactions explains this. Having transaction inputs refer to wtxids of previous outputs (even if that were possible in a compatible way) would defeat the purpose, as it would effectively be how things were done prior to segwit.
That does not explain why a wtxid exists at all though. An alternative could have been here to keep using txids everywhere. A problem is that this would break the ability to attribute errors to a block. Imagine in this design a malicious node that receives a (valid) new block, but sends it on to its peers with the witness data modified (e.g., invalid signatures). The victims would receive and start validating the block, and fail, but unlike today, they could not attribute the error to the block. They received an invalid version of it, but if someone else offered them the same block again, they would have reprocess it, because there is no guarantee that no valid version exists.
This is why wtxids were needed, and the reason they were introduced. Having them inside blocks (through a Merkle tree in committed to in the coinbase transactions) means that verifiers know they see the exact same witnesses as everyone else. If they're given an invalid block, they can just mark it as permanently invalid and never process it again if offered. Blocks are expensive to produce (proof of work), and thus this is an extremely effective DoS resistance technique.
How does this distinction affect the mempool and transaction relay?
When segwit was designed, the expectation was largely that txids would be used everywhere in the P2P protocol, and wtxids would just be used to commit to transaction data in blocks to address the attribution problem mentioned above.
This turned out to not work well. For various complicated DoS-protection reasons in transaction relay, that too (like block relay) benefits greatly from using wtxids. In particular,
BIP-339
introduced a way to advertize and request transactions by their wtxid instead.
If Bitcoin were designed from scratch, without its historical legacy or compatibility with existing software and rules, I think wtxids would be thought of as the "real" txids and used ubiquitously, and
only
prevouts would use a special "witness-stripped txid" that we call txid today.
Why does the witness merkle tree in a block use wtxids while the normal merkle tree uses txids?
If we could, the block Merkle tree would commit to the wtxids only. Unfortunately, that is not backwards compatible. Changing this would be an invasive hard-forking change that would require nearly all Bitcoin software anywhere to change.
For that reason, the old existing txid tree was left alone, and a secondary wtxid tree was introduced (through a commitment inside the coinbase transaction).
Quote
Report
Post Reply
Quoted from Forex.com.bd-Editorial External answer — Bitcoin Stack Exchange Author: Pieter Wuille Source score (net votes, not local likes): 4 Original post: https://bitcoin.stackexchange.com/a/131079 License: CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/ Adaptation: HTML converted to plain text; contact email addresses removed. This is a great question. The answer depends on which aspect you're talking about. To start, it is important to highlight what segwit's goal was. Besides being backward compatible (a softfork) and increasing block capacity it is most known for, its primary purpose was fixing the transaction malleability issue . Prior to segwit, chains of unconfirmed transactions were impacted by the inherent malleability of Bitcoin transactions. When two or more parties negotiate a set of dependent to-be-broadcast-later transactions (as in common in many layer 2 and off-chain protocols), those transactions and their validity depends on the txids of the outputs they are spending. As signatures and many other aspects of transactions (see BIP-62 ) are malleable, this breaks those dependent transactions as the txid they refer to can change before inclusion in the chain. This was solved by moving the malleable parts of a transaction to a separate area (the witness), so that txids are not impacted by it. What specifically would break if Bitcoin used only wtxid everywhere? Why do transaction inputs continue to refer to previous outputs using txid:vout rather than wtxid:vout? The desire to solve transaction malleability breaking chains of unconfirmed transactions explains this. Having transaction inputs refer to wtxids of previous outputs (even if that were possible in a compatible way) would defeat the purpose, as it would effectively be how things were done prior to segwit. That does not explain why a wtxid exists at all though. An alternative could have been here to keep using txids everywhere. A problem is that this would break the ability to attribute errors to a block. Imagine in this design a malicious node that receives a (valid) new block, but sends it on to its peers with the witness data modified (e.g., invalid signatures). The victims would receive and start validating the block, and fail, but unlike today, they could not attribute the error to the block. They received an invalid version of it, but if someone else offered them the same block again, they would have reprocess it, because there is no guarantee that no valid version exists. This is why wtxids were needed, and the reason they were introduced. Having them inside blocks (through a Merkle tree in committed to in the coinbase transactions) means that verifiers know they see the exact same witnesses as everyone else. If they're given an invalid block, they can just mark it as permanently invalid and never process it again if offered. Blocks are expensive to produce (proof of work), and thus this is an extremely effective DoS resistance technique. How does this distinction affect the mempool and transaction relay? When segwit was designed, the expectation was largely that txids would be used everywhere in the P2P protocol, and wtxids would just be used to commit to transaction data in blocks to address the attribution problem mentioned above. This turned out to not work well. For various complicated DoS-protection reasons in transaction relay, that too (like block relay) benefits greatly from using wtxids. In particular, BIP-339 introduced a way to advertize and request transactions by their wtxid instead. If Bitcoin were designed from scratch, without its historical legacy or compatibility with existing software and rules, I think wtxids would be thought of as the "real" txids and used ubiquitously, and only prevouts would use a special "witness-stripped txid" that we call txid today. Why does the witness merkle tree in a block use wtxids while the normal merkle tree uses txids? If we could, the block Merkle tree would commit to the wtxids only. Unfortunately, that is not backwards compatible. Changing this would be an invasive hard-forking change that would require nearly all Bitcoin software anywhere to change. For that reason, the old existing txid tree was left alone, and a secondary wtxid tree was introduced (through a commitment inside the coinbase transaction).
Checking account access…