A Comprehensive Overview of Bitcoin and Blockchain Technology

To help myself and others understand the complex cryptosystem and currency which is Bitcoin, I have published my Obsidian journal online. This journal guides the reader through technical explanations of the essential topics required to truly understand modern cryptocurrencies and blockchain technology. My notes and explanations are composed of various sources ranging from my own thought experiments to expositions from Bitcoin’s leading researchers and engineers.

The majority of the notes I have taken come from Andreas M. Antonopoulos’ book Mastering Bitcoin: Programming the Open Blockchain. This is the best book for engineers and computer scientists interested in actively working with blockchain technology and Bitcoin systems. In order to facilitate the learning process for myself and other readers, I have written my entire journal in LaTeX format and include, where possible, visual representations of the abstract concepts discussed.

The journal is split into multiple sections (i.e. guides), each of which isolate some central aspect of Bitcoin technology, and discuss it in detail. Through the following links, you will be redirected to my Obsidian journal. Please note that each of the guides build on concepts touched upon in the previous guides, so if you are new to Bitcoin and blockchain technology, I would recommend to read them in the following order.

Introduction and How Bitcoin Works

Powered by the technology of blockchain, Bitcoin changes how currencies can operate by defining a new system where authentication and authorization is decentralized. This achievement of decentralization allows for a distributed ledger that ensures transaction transparency and immutability and eliminates the need for centralized control or intermediaries. By operating on cryptographic principles within this decentralized network, these currencies enable secure, pseudonymous exchanges which are further enabled and secured the more people use this currency.

This link will take you to the corresponding guide covering the following topics:

1. History of Digital Currency and the Distributed Computing Problem

2. Constructing a Transaction with Inputs and Outputs

3. Adding a Transaction to the Ledger and its Propagation

4. Purpose of Bitcoin Mining

Keys and Addresses

Bitcoin’s keys and addresses are based on asymmetric cryptography, where a private key enables spending funds, and its corresponding public key allows others to verify transactions. With these keys, Bitcoin addresses are created by hashing the public key with SHA-256 and RIPEMD-160, and encoding the result using encoded representations via Base58Check designed to reduce errors. The private keys can then be stored in Wallet Import Format (WIF), a Base58Check-encoded representation, which simplifies importing keys into wallets. Keys can further be compressed to reduce public key sizes and enable more efficient address and transaction storage. Bech32 and Bech32m are newer address formats designed for SegWit which offer enhanced error detection/correction, improved efficiency, and easier QR code encoding.

This link will take you to the corresponding guide covering the following topics:

1. Asymmetric Cryptography and Private Keys

2. Generating Public Keys with Elliptic Curves

3. Hashes and Address Creation

4. Base58Check Encoding

5. Wallet Import Formats

6. Key Compression

7. Vanity Addresses

8. Bech32 and Bech32m Addresses

Digital Wallets

Wallets are digital tools used to store and manage cryptocurrency assets, with their design falling into two main categories: deterministic and non-deterministic. Hierarchical Deterministic (HD) wallets use a single seed to derive a tree-like structure of keys to enable convenient key management and backup through predictable key generation paths. This process often incorporates mnemonic codes, a human-readable sequence of words representing the seed, which helps facilitate secure and easier recovery of funds.

This link will take you to the corresponding guide covering the following topics:

1. Wallet (Non-)Determinism

2. Hierarchical Deterministic Wallets and Key Derivation

3. Seeds and Mnemonic Codes

Transactions

Bitcoin transactions transfer value by consuming Unspent Transaction Outputs (UTXOs) from previous transactions as inputs and creating new outputs for recipients, following a standardized serialization format for network transmission. Each transaction specifies a fee, which can be static or dynamically calculated based on network congestion and transaction priority. Within these transactions, the Bitcoin Script language enables programmable conditions for spending. Allowing for such “programmable money” enables Bitcoin to be used in a variety of different use cases. Historically, however, challenges like circular dependencies and malleability, where third parties can alter transactions without invalidating them, were intrinsic to this currency’s programmability. These issues have been addressed with improvements like Segregated Witness (SegWit), which separates signature data from the main transaction, and thus fixes the aforementioned issues while also introducing a new serialization format that improves scalability.

This link will take you to the corresponding guide covering the following topics:

1. Details of Transaction Inputs, Outputs, UTXOs, and Serialization

2. Static and Dynamic Transaction Fees

3. The Script Language and Transaction Scripts

4. P2PKH and Signatures in Transactions

5. Sequence Numbers

6. Circular Dependencies via 2nd and 3rd Party Transaction Malleability

7. Segregated Witness and its Serialization

Bitcoin Network

The Bitcoin network operates as a decentralized peer-to-peer (P2P) system where nodes communicate to validate and propagate transactions and blocks, with node types including full nodes, lightweight nodes, and specialized miners. These nodes communicate with efficient techniques such as e.g. Compact Block Relay which transmits only the differences between nodes' existing data to reduce bandwidth usage during block propagation. The Bitcoin network can further be segregated and classified into different network types including public mainnets and private testnets. Nodes within these networks discover each other dynamically using network discovery protocols, while technologies like Bloom Filters and Compact Block Filters allow lightweight nodes to request relevant transactions efficiently to preserve privacy while minimizing resource consumption.

This link will take you to the corresponding guide covering the following topics:

1. The P2P Network and Node Types

2. Compact Block Relay

3. Network Types

4. Network Discovery

5. Bloom Filters

6. Compact Block Filters

Authorization and Authentication

Authorization and authentication in Bitcoin ensures that only legitimate users can spend funds via cryptographic conditions defined by scripts. Scripted multisignatures allow transactions to require cooperation from multiple keys with the goal of improving security and enabling more complex use cases. Pay-to-Script-Hash (P2SH) simplifies these setups by encapsulating complex scripts into a single hash, and it can even embed Segregated Witness (SegWit) for reduced data overhead. Transaction timelocks enforce temporal constraints, specifying when funds can be spent, while Script flow control mechanisms guide execution and validation of conditions. SegWit formats like Pay-to-Witness-Public-Key-Hash (P2WPKH) and Pay-to-Witness-Script-Hash (P2WSH) streamline transaction verification by separating the signature data from the main transaction. Taproot and Tapscript extend Bitcoin’s functionality and privacy by combining multiple spending conditions into Merklized Alternative Script Trees (MASTs), hiding unused branches of the script.

This link will take you to the corresponding guide covering the following topics:

1. Scripted Multisignatures and P2SH

2. Transaction Timelocks

3. Script Flow Control

4. P2WPKH and P2WSH

5. Embedding Segregated Witness inside P2SH

6. Merklized Alternative Script Trees

7. P2C and Cooperation from Multiple Keys

8. Taproot and Tapscript

Digital Signatures

Digital signatures are a core component of Bitcoin designed to ensure the integrity and authenticity of transactions by cryptographically linking them to the signer's private key. The digital signature function typically uses the Elliptic Curve Digital Signature Algorithm (ECDSA) with signature hash types determining which parts of a transaction are included in the signature. Schnorr signatures, recently introduced in Bitcoin, provide improved efficiency and security through better batch verification, simpler multi-/threshold signature schemes, and proven security. Using the Fiat-Shamir transform, Schnorr signatures convert interactive identification protocols into non-interactive digital signatures. Script-based multi-/threshold signatures allow for public, complex spending rules, while Schnorr-based scriptless signatures reduce on-chain footprint and enhance privacy through Shamir-type verifiable secret sharing.

This link will take you to the corresponding guide covering the following topics:

1. Digital Signature Function

2. Signature Hash Types

3. Schnorr Identity Protocol and Signature

4. Fiat-Shamir Transform

5. Script-Based Multi-/Threshold Signatures

6. Schnorr-Based Scriptless Multi-/Threshold Signatures

7. ECDSA

The Blockchain

The blockchain is a linked series of blocks, each containing a header with identifiers like the previous block’s hash and a Merkle root summarizing transaction data. Blocks are connected through cryptographic hashes, forming an immutable chain designed to ensure the integrity and order of transactions. Merkle trees organize transaction data within a block to enable efficient verification by allowing nodes to check transactions without downloading the entire chain. Test blockchains, such as Bitcoin’s testnet and regtest, provide controlled environments for experimenting with blockchain features and developing applications without impacting the main network.

This link will take you to the corresponding guide covering the following topics:

1. Block Structure and Identifiers

2. Linking Blocks in the Blockchain

3. Merkle Trees

4. Test Blockchains

Mining and Consensus

Mining is the process through which Bitcoin ensures decentralized consensus by combining economic incentives with cryptographic validation to create and verify new blocks. Miners compete to solve computational puzzles and can earn rewards in the form of newly minted bitcoins and transaction fees. Mining involves assembling transactions into a candidate block, selecting valid ones, and ensuring the block adheres to the network’s consensus rules before broadcasting it. Miners often collaborate through mining pools where they share computational resources and distribute rewards proportionally by their contributions.

This link will take you to the corresponding guide covering the following topics:

1. Economics and Currency Creation

2. Decentralized Consensus and Verification

3. Coinbase

4. Mining the Block

5. Assembly, Selection, and Validation

6. Mining Pools

7. Hashrate Attacks

8. Changing the Consensus Rules