Breaking Down the Blockchain
(Without One Mention of Bitcoin)
Blockchain is being talked about a lot right now. But putting hype aside, what is blockchain? Blockchain is an open distributed ledger technology that represents a brand new type of application infrastructure and is largely thought by many to be the next big business disruptor because of its perceived democratizing potential.
As a software developer, blockchain was just an annoying buzzword to me until I took a little time to get past the jargon and the cryptocurrency craze to learn the basics of the technology. Let’s walk through some of the technical components of a blockchain (fun stuff like digital fingerprints, hash puzzles, and Merkle trees!) and break down how each piece contributes to this technology.
What is the Blockchain?
Let’s put together a human language definition of blockchain and a quick explanation of how it functions to start understanding how it solves the problems inherent in centralized transaction systems.
At its core, a blockchain is a record of transactions. It’s a database (or ledger) that is immutable and append-only — transaction records can be added, but they cannot be edited or deleted. Any data saved to the blockchain cannot be changed. By design, a blockchain operates not as a centralized system, but instead as a decentralized and distributed network of computers that are all connected by way of the internet — what we call a peer-to-peer network. All members of the network each keep an identical copy of the database and they all collectively agree to follow a set of rules for collecting, verifying and adding new data.
For a quick explanation, let’s consider a simple example where we track ownership on a blockchain. Think of a transaction as really just a text file with receipt information. It lists the buyer and the seller, how much is changing hands, and there’s a timestamp to capture when the transfer is effective.
Transaction data for multiple transactions are then packaged up into a block, and the block is sent out to each computer in the network as a message. Each computer (or node) in the network, follows the defined rules of the blockchain protocol and verifies the validity of the new block.
If the block is found to be valid, it’s added to the blockchain, which is very basically a continuously growing linked list of transaction data that represents the entire history of transactions for the particular asset being tracked.
Because each node keeps a copy of the blockchain, in effect there’s a consensus — an agreement — on the history of transactions. Each node acts as an independent witness testifying to the truth of the entire transaction history. As long as a majority of the nodes agree on the true copy of the blockchain, it is valid and can be trusted to be a truthful, legitimate ledger.
That was a very quick explanation, but it shows how the architecture of the blockchain solves for the problems of centralized systems. Because the blockchain is orchestrated using a distributed peer-to-peer network, there is no central authority that manages the record keeping. There also is no single point of failure in this distributed system, since each node keeps a copy of the blockchain.
All that said, there really isn’t one universal definition of blockchain. You’ll hear it talked about as a distributed computing architecture, you’ll hear it talked about as a special kind of database, you’ll hear it talked about as a chain of blocks. If you keep in mind three basic aspects of the blockchain design pattern though, you’ll be in pretty good shape.
- There is a network of computers connected peer-to-peer style. If your computer is part of a blockchain network, it is talking directly to other computers on that network, not through a central server owned by a central third party.
- There is a consensus algorithm. The connected computers in the network all run some version of the same software, and there are rules written in that software that define how all the nodes stay in sync and agree with each other on the transaction data that they’re saving to the blockchain.
- And, there is the blockchain data structure. It breaks down to a hash-linked chain of blocks…essentially a really long, kind of fancy linked list.
What we’ve been talking about above is a public blockchain model, but it’s not the only way to build a blockchain — there is also a private model. In the public model, anyone — you, me, someone halfway across the world — can read data on the blockchain and write data to the blockchain. In the private model, only those with authorization can access the network, so there is an access control layer built into the protocol.
If you add the idea of permissioning, that gives us four blockchain versions to choose from — public and permissionless, private and permissionless, public and permissioned, and private and permissioned. Public and permissionless is the most ambitious type as it’s a completely decentralized model. In practice, the other three combinations are what we see more of when the use case is not cryptocurrency related.
How Does the Blockchain Do What it Does?
This is the point where you might say, “Sure, great, sounds like a really good theory…”But it seems we have some questions to answer still:
- How do we know that transaction records on the blockchain do not ever get changed?
- How do we know that the parties of a transaction are whom they say they are and have agreed to the transaction?
- In a distributed system with many nodes, which are computers and which are users (read: actual humans), how can we verify each other?
To find out, let’s get into the weeds for a bit, talk about some of the technical stuff, and start building some mental models.
Cryptographic Hashing & Merkle Trees
Cryptography is a core technology that blockchain makes heavy use of. You’re familiar with cryptographic hash functions if you’ve ever used Git to handle version control for digital files.
Any amount of data, from a single letter to a directory of files, can be used as input to a hash function, and the output will be a hash (a fixed-length string of letters and numbers) that can be used as a digital fingerprint to uniquely represent that set of data. The output will always be the same for any identical set of data, but even a single change — a different letter or number, punctuation, or an added space — will result in a completely different hash, giving us a different, unique digital fingerprint.
And a very important piece of how the algorithm works is that it is essentially impossible to predict how a change in input will affect the output. So, each transaction being recorded on the blockchain is run through a hash function and can then be represented by and identified by that hash.
Let’s consider that the asset we’re tracking will be transacted many times each day, so we want to package up multiple transactions into a single block to be efficient. Since you can hash any data set — say, two hash strings — it’s possible to take a collection of hashes, representing multiple transactions, and run those through the hash algorithm. So, we hash each individual transaction up, then take those hashes, and hash them up as groups.
What we’re creating is a data structure that is referred to as a Merkle tree. It’s a type of binary tree built with hash pointers and it’s a very useful structure for grouping distinct pieces of data and making them accessible via a single hash reference. The one final hash representing all of the transactions is our root hash — it’s the Merkle root of our Merkle tree — and the root hash is a representation of all the transactions that are wrapped up in it.
Each block in our blockchain is going to have a collection of transactions represented by a root hash. The block header is where metadata for our block will live, so we’ll put the root hash in our block header, along with a timestamp indicating when the block was created, and we’ll include something called a nonce, which is just a number value. (We’ll come back to what this actually represents in a bit.)
Now, we talked about the blockchain being a hash-linked chain of blocks. How the blocks get linked together is by including in each block the hash reference of the previous block in the chain. Because a block ends up being just a piece of data, we can run it through the hash algorithm and get a hash that represents the entire block. To link our block to the chain, we include the hash reference of the previous block as a pointer to it. The next block created after ours will add a reference pointer to our block. And that’s how we end up with a hash-linked chain of blocks!
But, I didn’t really answer our question yet — How can we verify that any of these transaction records do not ever get altered? It becomes very easy to answer if we zoom back in for a second.
Because we know that changing even a single character in a data set will completely change its digital fingerprint, we can be sure that if a number or name is changed on the transaction record, the hash for that transaction changes completely.
And when we think about what that does to our Merkle tree of transactions, we know it will change the hashes all the way up to the root. If our root hash is different, our block hash changes, which invalidates the entire block. If our block hash changes, the reference pointer from the next block in the chain breaks, which breaks references all the way up the chain.
Depending on how far back in history this transaction that someone is trying to change originally occurs, it could break quite a bit of the entire chain.
Now, even if this person acting maliciously does the work to change all of those broken hashes and reference pointers in the Merkle trees and in the block headers, remember that we’re talking about a network of computers that all have a copy of the chain. This bad actor can only change the copy of the blockchain that they have on their computer, which results in a blockchain history that doesn’t match the version of the chain that all the rest of the nodes have all verified and collectively agreed on.
It’s a pretty simple yet masterful way to ensure that the transaction history of the blockchain can never be altered without it being completely obvious. Ensuring that one standalone version of the blockchain matches the true history of the blockchain is as simple as comparing two hashes, two strings of data, for equality.
Digital Signatures
Let’s take a look at our second question: How can we verify that the parties of a transaction are whom they say they are, and have agreed to the transaction?
Public-key cryptography is another core blockchain technology that is used to ensure each party in a transaction is whom they say they are; and that only they are the legitimate owner who has access to spend or sell their assets.
Public-key cryptography uses digital keys to protect data. As a quick explanation, each person is issued two keys. One key is identified as the public key, one is identified as the private key. The public key can be shared with anyone, there isn’t a need to keep it secure; but the private key needs to be kept very secure and needs to be kept in your possession alone.
The private key can be used as input to a function that will encrypt data — for instance, our transaction data. The output (which is referred to as ciphertext) is unreadable by both humans and computers. Only the twin public key can decrypt that ciphertext and turn it back into readable data. The idea is that the seller in a transaction uses their private key to encrypt the transaction data and they add the resulting ciphertext to the transaction as a sort of digital signature.
The seller’s public key is then included in the transaction and the buyer uses that key to decrypt the signed transaction. If the decryption is successful, the parties can be sure that it was truly the seller who both initiated and authorized the transaction.
Each transaction that gets broadcast is digitally signed, and the public key — the only one that could ever successfully decrypt the included ciphertext signature — is part of that transaction data. As long as private keys aren’t stolen or compromised, this gives us a very reliable way to ensure sellers in a transaction are whom they say they are.
Hash Puzzles
And finally, the biggest question of trust — In a distributed system comprised of many nodes, how can we verify that each peer is reliable and trustworthy?
The answer? Hash puzzles.
One benefit of a centralized software system is that there is generally a manageable number of nodes which need to be secure, and a manageable number of trusted members of the network. Conversely, that means that one of the disadvantages of a distributed, decentralized system — with a much higher number of nodes — is that there is a higher probability that some of those nodes are untrustworthy.
Because the benefits of a decentralized system are what make the blockchain a viable idea in the first place, the problem of how to achieve and maintain trust in a decentralized system was one of the biggest problems that needed to be solved in order to make the technology viable. This was done using the “Proof of Work”concept.
Proof of Work is a term that refers to a piece of data that is very difficult to produce (meaning it is costly and time-consuming — computationally, a very heavy task), but is very easy for others to verify. The idea is that in order for a node in the network to be able to successfully produce a valid block of transaction data, they need to satisfy the Proof of Work requirements before the block can be accepted and added to the blockchain history. And in practice, Proof of Work is implemented as a hash puzzle.
Back to our transaction — the blockchain protocol states that each new block created needs to meet a certain difficulty level. The difficulty level translates to meaning that the hash that represents the block needs to meet a hash target — it has to take a certain form. The hash may need to begin with a string of four zeros, for example.
Since we know that it’s impossible to predict an outputted hash string, the puzzle involves adding a number to the block data, hashing the block, and checking to see if the resulting hash meets the target by having a string of four zeros at the beginning of it. If it doesn’t, the number is incremented and hashed again. The process repeats until the number we’re adding — the nonce — gives us a hash that meets the hash target.
For a quick example, if we’re working with the string “Hello, world!” and we add a zero to it, we get a hash beginning with 1312af. If we change it to a 1 it’s completely different, if we change it to a 2 it’s completely different again, but still no zeros. If we continue incrementing, it’s not till 4250 do we get a hash that starts with 4 zeros (0000c3). So, 4250 would be our nonce, and basically the answer to the puzzle.
If we get back to our transaction, it’s the very same idea — and since we can’t in any way predict what the hash output will be, it’s just a big guessing game for the computer. The only way to find the right nonce and solve the puzzle, is to brute force it, to test the number one by one.
Here is our transaction with a nonce of 0, and a hash beginning with BA49does not meet our hash target of 4 zeros at the beginning.
Incrementing to 1 doesn’t work, and incrementing to 2 doesn’t work either.
The idea is, it’s going to take a really long time to find the random number that meets our hash target.
This is a very computationally expensive exercise, especially at higher difficulty levels — it takes the computer lots of time to compute, computation requires energy to power the computer, and energy costs the user of that computer real money.
So, the Proof of Work concept plays an extremely important role in the blockchain implementation because it means that verifying transactions and adding a new block to the chain is a computationally expensive exercise. It forces the user to “have some skin in the game” by costing them real energy and money. And if we also reward the node who solves the puzzle first with an economic incentive to do all this computation, that’s a very strong deterrent that keeps a node from trying to alter a transaction or a block. It has a reason to play by the rules and remain a trusted member of the system, which helps enforce the integrity of the distributed peer-to-peer network.
Where and When Can Blockchain Help?
So where and when does it make sense to explore using blockchain technology? Let’s start by mentioning where it ISN’T needed — and if you only take one thing away from this blog post, make it this:
If your requirements can be satisfied by a relational database, you don’t need a blockchain. Blockchains are a technology for shared database scenarios where you can’t necessarily trust all the entities that will have access to it.
If you have an idea where a blockchain could be an interesting solution, but it would have to be private, and probably permissioned, think critically about whether or not a blockchain is really the right solution.
There are though certainly areas where blockchain can help, and maybe even revolutionize the status quo. Interesting application are being explored in transactions and digital assets, health records, identity documentation, supply chain and product lifecycle, city services, Internet of Things, cryptocurrencies, and digital games such as CryptoKitties.
Curious what’s next for blockchain technologies? Read about blockchain, and then read some more about blockchain, and afterwards read some more. You’ll find opinions on this new technology are all across the spectrum. Some are calling it the second coming of the internet, some are calling it shoddy technology that would lead to a lawless and mistrustful world, and everything in between. Read lots of perspectives and decide where you and your potential use cases land. Blockchain is also a very deep topic and gets even more deeply technical the further you poke around under the hood.
Related
To help connect some of the dots, here are some resources I’ve found very helpful:
