Pull the chain – The Blockchain

 

Blockchain

Blockchains are everywhere at the moment, with pretty much every large tech company investigating whether they can practically use them for… well… “something, there must be something we can use them for?”.

So are Blockchains a solution looking for a problem?  Or will they be truly revolutionary beyond the hype around virtual coins?

Here is everything you need to know about Blockchains, where they came from, how they work, and some of the potential use cases.  Sit back, pop a virtual coin in your nearest digital beverage machine and let’s get to the bottom of it..

First Blockchain – what is it and where does it come from? 

Blockchain was invented by Satoshi Nakamoto in 2008.  Who is Satoshi?  Well either a person or a group of people who developed the concept for use in the crypto-currency Bitcoin.  Nobody really knows to be honest, or if they do they are not telling or proving it.  Suffice to say it is not important to get a grip on the technology.

So what is Blockchain?    As I just mentioned, it is the foundation for crypto-currencies, which, as John Oliver wisely said, are “everything you don’t understand about money, combined with everything you don’t understand about technology”.  An ideal opportunity for a hype cycle if ever I saw one.

Before going into the technology of Blockchain, let’s look at the concept of crypto-currencies, of which Bitcoin is surely the most famous (and the focus of the Satoshi paper), and then onto blockchain which is essentially the underlying technology that allows crypto-currencies to exist.

 

Crypto-currency

Let’s break this down.  Crypto – refers to cryptology (study of) or cryptography – the art or science of writing or solving codes (today, most associate this with mathematical techniques).  So we are talking about codes associated with a currency to perform some function.

Currency – a system of money in general use (typically within a country,  but in the case of crypto-currency there is no specific country associated with it, even though it’s worth is typically related to a more traditional currency like the dollar).

With that defined, let’s now ask a bigger question – why does money have value?  Not going into too much detail on this, but essentially it provides a unit of account or value making it easier to exchange stuff, a medium if you will.  Easier than bartering the value of goats for chickens or every item under the sun relative to one another.  It gives a stored value to be applied to each of these things.  The fun thing about money is it allows economic transactions over time, at any distance, for anything.  The other thing to remember is that money is only worth what someone else is willing to give you for it (be that silver, gold, coins, paper, or virtual coin). Physically producing more money does not make everyone richer, but it can affect its value.  If there is too much of it, it might be worth less, i.e. someone might give you less for a certain amount of it.  We see this everyday in economic transactions, central banks, markets etc.

What is it about Crypto-currencies that are so appealing at first glance?

When you consider a currency that is not controlled by any government, corporation, bank, or individual, and is essentially just code, then maybe you see its appeal.  When it first came on the scene it was viewed as a kind of virtual money Utopia.  No bank in the middle to transact regardless of geography.  There are technical challenges of course, but the concept itself is attractive.  As such it is known as a de-centralised currency, in that the records (or ledger) of every transaction ever made can be stored in a de-centralised manner across many computers anywhere (not like a physical ledger sitting centrally in a vault).   A distributed ledger, no central server, or repository of information, so better for security (no single place to attack), and potentially fast.

A secure, distributed database then.  Who could see an advantage in that?  Hence all large tech companies looking for uses around sharing data, securing transactions etc.  If it works reliably (and time will tell as it is still very early days), it looks like a solution that might be applicable to numerous problems, not just crypto-currency.

The scramble has begun.

It is the Blockchain technology that enables this so let’s take a look.

Blockchain 

What is it and how does it work?

The blockchain is the essential underlying technology that enables cryptocurrency.

A blockchain is a chain of blocks, and blocks are packages of information (a unit of data, value of money, anything really).  Each block has three things essentially: the data, the hash, and the hash of the previous block.  These are all linked together or ordered in a network where all the peers are not trusted.  The block represents the record of a transaction (or group of transactions)..

Alice wants to send Bob some money.  This transaction is represented as an input and output address with a value (which will be explained later), and sent or broadcast to every party in the network, who then approve this as a valid transaction.  The transaction is ultimately validated by being included in a “block”  that is added to the “chain” hence blockchain.  This is a transparent and indelible ledger or record of transactions.  The money flows and the transaction occurs between Alice and Bob.  Every transaction is listed and linked to all previous transactions, and is there for all to see as a distributed ledger.

Sounds simple enough on the surface, but what is really going on here, and how is a secure distributed ledger achieved?

Consensus

Consensus and proof of work together, are really the magic behind Blockchain technology (and Blockchain is the magic of crypto-currency)

Here’s a term for you, ‘Distributed Trustless Consensus’.  How to achieve this through cryptographic means is pretty much the major innovation of Blockchain.  From this term comes an understanding of transactions and makes mining easier to grasp.

Consensus – two parties (or indeed multiple parties) agree that something means something.  We both agree on the meaning, and in addition we both know that the other agrees.  With a currency this understanding or consensus not only applies to the transacting parties, but to everyone using the currency.  When it comes to money there needs to be a consensus that there is not an unlimited supply (scarcity), and that everyone agrees on it as an accepted means of value exchange.  What we are also implying is that there is a consensus around accepting these tokens as money of comparable value, and everyone sticks to it.  Once there is consensus, we can transact.

consensus

In the physical world we are all familiar with the money supply, and that it is not unlimited, but controlled by a central bank, usually country specific.  With digital money this can be mimicked by having a central database on a server which keeps track of who holds what money, and when transactions are performed so money is not overdrawn.  Like digital banking.  Where we are going with blockchain is towards a distributed database to achieve this, rather than centrally with today’s digital banking.

So this is consensus, and how you get there (either centrally or distributed) is a key concept underpinning blockchain.

First, achieving consensus with proof of work solves the infamous double spend problem explained below.

The double spend problem

In a central mint system, once a physical coin is replaced it goes out of circulation, is returned to the mint where possible (to be converted into new coin), and then not accepted as currency.  This stops the double spend problem as there is only one unique coin in circulation at any one time and therefore you cannot spend the same coin twice.  You can have 2 coins of course.  Say there are 100 x 10 pence coins in circulation.  You may own 2 of the 10 pence coins in circulation, but you cannot use the same coin twice, unless it has found its way back to you through a series of transactions – in which case it is a separate transaction and therefore not a double spend.  You can’t give the shopkeeper 10 pence for some sweets, take the 10p back and buy the sweets again, as you have already exchanged it for some goods or service, so now you are stealing that 10 pence piece.  As the 10 pence will have gone into the cash register, or the merchant’s hands, everyone knows you are trying to steal it back or double spend!

The UK’s national currency exists in 3 main forms, one is physical, the other two are electronic:

  1. Cash – banknotes and coins.
  2. Central bank reserves – reserves held by commercial banks at the Bank of England.
  3. Commercial bank money

With money represented online via your bank account, the commercial bank is the central point of control and the money is represented as currency in your account (e.g. pound sterling). There is still only so much money in circulation as far as the central bank is concerned (total physical cash, and total electronic cash held in the central reserves), but commercial banks themselves can actually create money too.  They do this simply through lines of credit or loans.  Whenever credit is issued or loans are deposited into a person’s account, the bank has literally created money.  This all then needs to be squared off through the financial system so that, should the economy take a turn for the worse, and people want their money withdrawn from a bank either into either physical cash or into another electronic store (not stored in a commercial bank, or even a safer commercial bank) there are enough reserves in the system to account for these economic shocks.   With the financial crisis it transpired that commercial banks were not holding enough in central bank reserves (money the central bank has “printed” which the commercial bank has acquired) to weather the economic storm.  When people wanted their money back through this interdependent chain of loans, ultimately at the end of the chain, the reserve money simply wasn’t there in enough quantity in commercial banks to cover their liabilities.

This all gets rather complicated, but serves to illustrate a point with digital coin, in that creating it is analogous to a central bank that “prints” new physical or electronic money to add to the total circulation.

Digital coin acts like a central reserve system in that if you want more physical currency, or a “note” added to the system, then in the central reserve world the total amount of currency in the system (physical money in circulation and electronic money in the reserve) needs to go through a process  –  namely printing the note, having it all approved, and the corresponding amount deducted from the electronic total in the central reserve.

So creating digital coin (like bitcoin) is similar to creating a central reserve printed bank note.  With a central reserve it is difficult to do, and there is a long central social process (with checks and balances) to make sure there is no double-spend when a new note is printed (i.e. the same bank note produced twice), whereas with digital currency there is a digital distributed proof or work process to achieve the same thing.

Bitcoin is both a store of value and a medium of exchange, and can act as a unit of account (i.e. related to some value in the real world, in the same way that the commodity price of gold could be compared to a weight of grain).  As it takes electrical energy to produce a block, it could therefore be compared to a watt-hour for the initial unit of account, and it would be to this unit that initial value can be applied.  Demand, of course, assigns any real-world comparative value to an existing currency, or an exchange rate.

The store of value is the IOU, and the medium of exchange are the linked transactions of the blockchain.

The trick is to make this a distributed system, and fortunately we know how to do this – through distributed databases.  The only question is whether all the records on all the servers are updated immediately for strict consistency, or there is a lag while everyone gets up to date, which is called eventual consistency.  With eventual consistency there needs to be a small safety net so transactions can be cancelled if things get out of synch, or a conflict elsewhere on the network to get things lined up again.  As long as this can be achieved and everyone agrees, we have ourselves a system.

Now we have explained the concept of consensus and how it roughly relates to existing currencies, we need to look at how this is implemented in a distributed system through proof of work.

Byzantine Generals Problem

Constantinople

A good lead-in to proof or work, and framing of the distributed system problem it solves around consensus, is a very brief explanation of the Byzantine Generals Problem.

Say there are two armies who need to work together to invade an armed city, and they are sitting ready and waiting on opposite sides of the city.   The Generals of each army would like to agree an attack day and time, as they know if they both attack at the same time they will win, but will certainly lose if only one army attacks.

General A wants to send a message that says to General B, “agree to attack at 4pm on Thursday?”  General B then replies  “Thursday is bad as that is the day we hold our army’s mindfulness yoga session.  How about Friday at 12pm?”  General A replies “ok, agreed”, and General B replies saying “Sorted then, agreed”.

In order to get the message across, each General dispatches a lieutenant with the messages.  The problem is the messenger travels through the city to get to the other side and could be compromised, so how do you believe the message? Or what if the message gets changed to “agreed Thurs at 4pm”?.. meaning only one army attacks and they get slaughtered.  This extends with more parties and more messages so there is a need to ensure the integrity of all the messages, and timings.

In our Blockchain story this is where the linked hashes and proof of work come in, so keep this Byzantine Generals Problem in mind as we walk through the components and onto proof of work and timestamps.

Blockchain uses eventual consistency to achieve consensus and achieves this through proof of work.  In order to step our way to proof of work, let’s see how the Blockchain is built up and how the underlying technology works – this makes proof of work simpler to understand.

The meat and potatoes – how the components come together

What we are trying to achieve is consensus around the validity of the transactions and the record of transactions, namely the ledger.  For this the crypto hash is a very handy tool.

Hashtag #

What is a hash?  Think of a hash as a one-way scrambler, where you get the same scrambled egg pattern every time you scramble that specific egg.  Any other egg would produce a different pattern, but the same individual egg will always produce the same pattern of scrambled egg.  This mathematical magic is very useful as it can provide a virtual fingerprint.

Technically with a hash you can take an arbitrarily long string of bits and produce a fixed sized result, where it becomes unfeasible to construct two different strings or messages that hash to the same value.   Good hashes require properties such as being truly one-way, so once scrambled it is pretty much impossible to tell what the egg looked like in the first place.  It can be computed but not inverted.  Secondly it must have good collision resistance, which means that two messages that are different are exceedingly unlikely to produce the same hash value when put through the hash function.  It is this property that makes hash functions useful in signature schemes.  If you have a value or word like “blockchain” and put it through a hash, then the hash value might end up at the number 241, (for simplicity’s sake – the real values are much bigger of course), but if you add an ‘s‘, then the word “blockchains” might end up at the value of 162.  Different messages should never end up at the same value, as that is known as a hash collision.

This way, if anything changes about a string or value you are interested in, then it produces a different hash value.  This way you know it has been altered.  Equally, if the value stays the same according to the algorithm (it is therefore what you expect), you know it has not been altered.  If I put the word “blockchain into the same hash algorithm, I will get the same value every time, and nothing else will ever produce that value.


Actual Sha-256 Hash of “blockchain”

EF7797E13D3A75526946A3BCF00DAEC9FC9C9C4D51DDC7CC5DF888F74DD434D1

Sha-256 Hash of “blockchains” 

Result = completely different fingerprint

99CF6497AFAA87B8CE79A4A5F4CA90A579773D6770650F0819179309ED846190


How does all of this relate to a chain of transactions?  Well the humble hash is used as linking method from one block to the next, a bit like a linked-list.

Blockchain as a kind of linked list

For those who have done a bit of Computer Science or coding, and know your data structures, you will be familiar with the concept of a “linked list” – a data structure of no fixed number of elements which allows insertion and deletion in a less expensive way than, say, arrays.  It took me a little while to get my head around coding the various forms of linked list when I was at University, but the concept is simple enough, and no need to code them for this explanation.   You can see the association with blockchains in the ‘link’ between elements (see below).  In a linked list you have the data element, and then a pointer to the next element.

Linked-list-1

This is similar to Blockchain except that each block has a hash of the previous block, which could be looked at as a pointer to the previous block.  The reason I have included this explanation is because you will often hear people saying Blockchain is basically a singly linked-list.

The data structure itself is a little more complicated than a linked-list and this is where Merkle trees come in

Merkle trees

Raplph Merkel of Berkeley and Stanford university patented the concept of hash trees in 1979, and a quick Google search will find his thesis on Merkle trees.  Ralph Merkle is famous of course for his contributions to both public key cryptography and the Diffie-Hellman key exchange, with Hellman in 2002 suggesting it should be called the Diffie–Hellman–Merkle exchange to recognise the contributions made by Merkle.

What is a Merkle tree?

Merkle trees are constructed bottom up starting, therefore, at the bottom by hashing individual transactions (typically a SHA-2 hash function).   You then add a timeline, whereby each node (or non-leaf node) is a hash of previous transactions up the tree (like a linked list with a pointer to the previous node).  As you are combining transactions you need an even number of leaf-nodes, so if the number of transactions is odd you duplicate the last hash to produce an even number of leaves.

Take a look at the below which might make it easier to understand.  Each transaction is hashed and stored in a node.  A node in the tree sense is simply  a point of intersection/connection within the tree (a node in Bitcoin is similar but involves a few other things described later when we come to users, nodes and miners).

Consecutive pairs or nodes are then hashed (e.g. A B), and summarised in a parent.  This keeps going until you have a hash of all the transactions summarised in a single root hash (ABCD).  If someone changes something anywhere in the tree, or it is not in sequence, the root has a different hash and you have an invalid block (remember the unique properties of a hash fingerprints mentioned earlier).

So ultimately a Merkle tree can summarise all of the transactions in the tree in a single fingerprint or hash.  Powerful stuff.  A user can now look at a transaction and verify whether it is included in a chain of blocks, or chain of transactions.

 

Merkle

Through this method you can now prove whether a log or ledger is consistent – a general distributed ledger.   In each instance you need to verify that no previous records have been tampered with, namely the integrity and validity of the previous data.  Simple answer, compare the root hash and away you go.  All data or logs are stored chronologically, so any later versions simply need to check out with root hash to be considered valid.  This shows whether any previous versions of the ledger have 1) all the data/transactions recorded, and 2) whether chronological order of the log has been maintained.

One other thing to note, as opposed to a hash-list, is that a branch of a Merkle tree can be downloaded individually, and therefore the integrity of individual branches can validated immediately.  This means small data blocks can be downloaded and you don’t have to download large entire data blocks if the original has become damaged somehow.  Therefore only tiny amounts of information need to be transmitted over the network, thus proving integrity and validity is fast and doesn’t take up much disk space or compute power.

Merkle trees then, in short, help prove that a log or ledger is complete and consistent.  Nothing has been altered or tampered with, nothing added, and the tree hasn’t been branched or forked.

 

The BLOCK

Silver_Block

Think of a block as the thing added to the chain that enables new coin to be awarded and the currency to expand (adds coins or currency into circulation).  Once a block is created by winning a puzzle competition, coin is deposited into the winner or miner’s wallet (created), and then into circulation to be used for transactions thereon in.  It also validates transactions.

A block essentially consists of:  a  unique identifier to the block (this is the result of the mathematical puzzle), previous block reference, the list of transactions, and a nonce or random guess.

A block (or group of transactions bundled into a block), represents a value of virtual coin effectively (confusing isn’t it).  Once a block is created and added to the network, coin has been produced.  The blocks have been given a value as some work is needed to be done to produce them, and they are unique.  It is the blocks that are theoretically worth something as there are only so many blocks in the system (like a currency – only so many 10 pence coins in the system).

Fundamentally if you have been successful at adding a block to the chain, you have added money into circulation, as you are rewarded with virtual coin deposited in your wallet for your efforts  (12.5 coins per block today).  This is the only way to add new coin to the currency.  There has to be a starting point for the currency and this is it.  From here you can exchange coin, give it away, use it in transactions etc. but you are not adding to the total currency in circulation from this point on.  You must wait another 10 mins for someone else to solve the puzzle and add another block for more currency to be added.

bitcoin__1__thumb800

Think of a “blockchain” as a list of all solved blocks.  What does that mean?  A solved block contains a list or ledger of valid transactions that have been completed.   When a transaction is published to be processed on the digital currency network, it goes onto a list of unconfirmed transactions waiting to be validated.  Miners then have to solve a hashing puzzle which will, in turn, “solve” a block and add it to the ledger.  Basically they will take a bunch of transactions on the “waiting to be approved” list, bundle them together and work out a hash.  This hash need to check out all the way up to the root.  If the chained hash checks out, the transactions are valid.

When Alice sends money to Bob, she hashes the transaction, creating a transaction-id.  At this point the transaction is not valid, but Alice announces the transaction to the digital currency network.  It then waits for a “miner” to validate the transaction.  Miners do this  by including the transaction in a “block” when they create a block of transactions. The transaction-id is a hash and linked to all previous transactions, and this feeds into a block which is a hash of a bunch of transactions which, when combined with other block hashes, form part of a hash tree all the way back to a root hash.  If this combination of linked hashes of all previous transactions is out of order or changed at any point, including block hashes, the chain is not validated and the chain not extended with a new block.

At this point we know the transaction is not fake and Alice indeed has £10 in her account and that the account can be reduced by £10, while Bob’s increased by £10.

Each block contains up to 500 transaction on average in the example of Bitcoin.

Transactions

If we are saying that virtual coins are merely a chain of digital signatures, therefore each link in the chain represents a coin value, then there has to be a way of tying them together, and making sure a previous owner of a transaction block has not already signed another block or coin (leading to double spend, or two coins in the system with the same characteristics signed by the same owner – like two £10 notes with the same serial number).  So in order to know of an absence of a Block in the system for the same thing signed by the same owner, we need to know about all the transactions.  The network needs to know about all the transactions.

This is quite difficult to do once the chain has started as the hashes are all linked, so we only need concern ourselves with the first transaction.  Each transaction is linked to previous transactions through a hash and in turn linked all the way back to the first transaction.

Components of transactions

Let’s now talk about transactions themselves, and frame this discussion by first understanding some of the component parts of the crypto-currency network, namely: users, nodes, miners, wallets, addresses and finally the transaction itself:

Consider that a crypto-currency network typically involves three main things:

  • Users enact transactions.
  • Nodes validate mined blocks making sure they follow the rules and participate in network consensus, as well as relaying blocks to other full nodes.
  • Miners are rewarded with new coin for the hard work they have done solving a puzzle in order to add a new Block to the chain and in turn validate a bunch of transactions.

With this in mind let’s see how this relates to wallets, addresses and the transaction itself.

The Wallet

wallet

The idea of a wallet with crypto currency is an important one and has an analog in the real world.  This terminology causes a problem as the real world analogy leads people to think of their physical wallet with cash or credit cards etc., but a Bitcoin wallet is essentially just a software program where addresses that are tied to coin values are stored.  Technically Bitcoins are not stored anywhere.  Instead there is a private key or secret number associated with every Bitcoin address saved in the wallet of the person owning the balance.  So in essence the wallet is a balance, but not in your traditional sense as there is no such thing as a transaction “balance”.

First, consider a wallet as a collection of private keys that correspond to addresses.  Addresses are tied to values of crypto coin.  Value in the Bitcoin world is created by miners who, upon solving the puzzle and winning the lottery-style competition, form a new block and are assigned coin or value for their effort.  This is essentially an address in their Bitcoin wallet.  Value is transferred from miners to others through the transaction process, so that other addresses can “own” some of this value.  The address corresponds to a value of cryptocurrency.

A wallet and an address are not the same thing.

wallet is a collection of private keys that correspond to addresses.  You need a private key to spend from an address.  These are typically stored in a file on disk, and have features such as encryption or address labeling.

What is an address?

An address is therefore a place to send crypto coin from or to.  An address is a set of arbitrary letters and numbers (digits) that represent a particular user’s balance.  You would own a wallet, within which you have addresses.  You can send and receive crypto-coin as many times as you like to other users, with your own address being the source.

An address is a Bitcoin public key to which transactions can be sent.  In fact it is a hash of a public key that can then be published.  So given the address is essentially a public key there is no problem in distributing this publicly.  I can advertise the address and anyone can send me coin.  If you open up your wallet and send coin to my address, as I am the only one with the private key I am the only one who can receive the coin.  If you want to make sure I am who you think I am, you could ask me to sign a message associated with the address with my private key and then verify with the public key.

You can see from the below diagram how electronic coin is defined as a chain of digital signatures.  You can also see how each transactions is linked to the previous transactions in the chain and where addresses fit:

Transaction process

 

The Transaction itself

transaction1

Transactions occur between users

When users transact coin there are 3 components.

  1. Transaction Input – which is the address (described above) from which the money was sent.
  2. Transaction output – the address to which the money was sent.  Remember the discussion on wallets?  This is the address in your wallet where you receive the money.
  3. Amount – which is the amount of money (coin) that was sent.

There is a transaction chain of events or a ledger, which means that any money you send to someone was originally received by you from someone else in the chain. These addresses are registered on the blockchain (public keys), so when coin was sent to you, that address was registered on the blockchain as a transaction input, and when you sent it your address was registered as a transaction output….and onward as part of a chain.

As all the transactions are recorded in the distributed ledger and chained, you can see how balances are always up to date and accurate, without a central authority or interconnected banking system acting as a point of control with their own ledgers to keep track of balances.  If there is a transaction input linked to an output and a value, then the balance of the addresses where money was sent to and from is easily recorded.  If you spend coin, the value of coin your address represents goes down, and the value of the address to which you send coin correspondingly goes up.  All recorded in the chained ledger.

Through the ability to check a transaction chain quickly you get to avoid the double spending problem, whereby if Alice pays Bob £10, you need to be sure that Alice hasn’t also used that exact same money to pay someone else.  If Alice has a valid transaction in the blockchain then the network can go ahead and update Alice and Bob’s account by subtracting and adding the £10.

This brings us on to the important topic of consensus by proof of work.  Proof of work which has been mentioned many times in the article, is the key to Blockchains.  That is, how to actually add a block to the network and then be rewarded for it with coin to increase the amount of coin in circulation, which can then be transferred from wallet to wallet through transactions.  Account to account transactions.

Bitcoin seems to be worth real money now  (sounds good), so say I want to participate and add another block to the chain, how can I do this?  Well there is work to be done…

Proof of work (or essentially proof of time)

Think of proof of work as a distributed clock that allows consensus on timings, making sure that all transactions or blocks (and new coins) are added to the crypto-currency correctly, in order, with no cheating or fraud.  This timestamp is at the heart of Blockchain so we are going to spend a little time on it.

Timing is everything.

Timestamp

Bitcoin mints new coin every 10 minutes, and it does this by starting the puzzle race.  Solving the puzzle is the proof of work, which conveniently also provides our timing and the distributed clock.

How are new coins created?  As I mentioned earlier there is no central bank controlling the digital coin money supply.  Instead there is a finite limit to the number of coins that can come into existence (in the case of Bitcoin this is 21 million coins).  How are new coins created?  What is stopping all these coins being created at once by one person and verified?

How do I get hold of digital coin?  Well you can buy or trade them, or you can actually mine for them (a bit like mining for gold – you get your tools out, find where to look, do some work and find some gold).

building blocks

Suppose you want to be a part of this puzzle solving network with your own computer resource (add blocks, get coin in your wallet, and make money!).  Well you can, and this is called mining.  If your compute power solves the puzzle to add the next block to the transaction ledger then you win!  You are then rewarded with 12.5 shiny new Bitcoin which now enter circulation.

But how is this actually done?

I have talked about Merkle trees which has blocks added to it linked by hashes, but there needs to be a way to actually create this store of value or block.  Once the block is created it must be related to previous blocks so it hashes out correctly.

So I need to create a block that somehow is allowed to contain the hash of a previous block and extend the chain.  Do I go and ask someone for this?  There is no-one to ask, so there must be a way of verifying that any new block is allowed to include the hashed link to an old block.  This is where proof of work comes in.  In order to create a new block I must solve a puzzle.

Essentially we give all the computers a puzzle to solve and start a race.  First you solve the puzzle, then once solved, in lottery style fashion, one of the puzzle solvers is chosen to add a new transaction to the blockchain.

The puzzle – Proof or Work

Rubik

You need to solve a puzzle, and how hard that puzzle is dictates how hard it is to produce Bitcoin or any other coin based on this method.  What we are essentially saying is that to create a new store of value (the block), we need to have it as part of the chain.

When a new Block is added to the chain by a miner, the network is provided with 2 hashes. The first is a hash of all the transactions in the block (as tied in with the transactions chains already discussed), and the second is a hash to prove that the miner has done a heap of work in order to create the block.  As the amount of work (CPU, energy etc.) is large it makes it unprofitable to do all this work under false pretences.

At each tick of the Blockchain clock a new Block is added, and for this there can be only one winner.  So it is part lottery and part puzzle.  Lottery in the sense that regardless of computing power you can get lucky and find the required hash result of the puzzle, but in the main it requires a good deal of computing power to get the answer.  Whoever gets the answer first is the winner and gets to add a block to the chain, then are rewarded with digital coin in their wallet.  Coin is added to the currency at this point.  The rate at which blocks are added is dictated by the clock tick, which in turn, is dictated by how long it typically it takes to solve the puzzle with lots of computers working on it.

So what is the puzzle?

A hash (as mentioned earlier) is a unique digital fingerprint. Put some information through a hash and you get a fingerprint for that information or data.

Take the following Sha 256 hash of the string “blockchaintest”,

you end up with:

45A0B443773F71D947B4313654FB77CDFE85F20DDF19D13358F345FDE7790A8E

Of course if you receive that hash value you have no idea what the original text was, or what it means, but if I ask you to hash the same original text you will get the same value using the same algorithm – a unique fingerprint for that text.

Now I give you a computational puzzle.  I could say, “add a number to the end of the string and hash the string.  When you have the first number that makes the hash result start with a ‘0’ you have won”.

So first you try:

“blockchaintest1”

FB273A360DE9E70FE471564B0340358EE6CBB3AAC230619D22A5AE3808DBE606

but no dice

Then you append each sequential number and retry, and in this case 25 is the first number appended that make the hash result begin with “0”.

“blockchaintest25” 

0411BA1CD1DB58798ECDC197FF9E6163707DA29CA226DCEB334407B116CF012B

What this tells me is that I have done 25 pieces of work in order to get to the result, so have expended computational energy.

As you can imagine this example does not take much computational power, but say I ask for the first number appended that makes the hash result begin with fifteen “0”s?  Now this takes some power and time to hit the number.  In effect I am asking you to guess the number that will produce a certain hash output, and that number is a nonce (number used once).  So in order to solve the puzzle I need to guess a number, the hash of which meets certain conditions.  The puzzle is to guess a nonce that satisfies this condition, and the level of difficultly of guessing the nonce is fixed across the network, so much so that it takes a typical length of time to guess (in Bitcoin this is 10 mins currently).  So every 10 mins a new block is added and therefore new “coin” is added to the currency which can then be transferred like money across the network (wallets, addresses, all that good stuff).

Everyone could start this puzzle at 0 and increment from there, but this leads to a definitive race condition where, whoever has the most compute power will certainly win every time.  You have the freedom to start guessing from wherever you like of course, and the puzzle remains sufficiently difficult wherever you start from.  Probability dictates that on average it takes a certain amount of compute power and hash guesses to solve the puzzle, so typically the entire blockchain network will make a successful guess once every 10 minutes, and whoever guesses correctly first within this network gets to add a block and make coin.  You therefore have a network of compute power that can throw as many random guesses at the problem as possible, which on average takes a certain amount of work, and whoever guesses first in the entire network gets to add a block.  Of course, if you are very lucky you can guess the nonce that produces the correct hash in double quick time with low compute power at your disposal,  but in general you need to be very lucky indeed as it is governed by probability.  This is a lottery style win to add a block.

The best bit about this system is it might take me millions and millions of pieces of work to get the answer to the puzzle, but once I have the hash and forward my answer it only takes one piece of work validate that it is correct.  It essentially proves that whoever wins has done a pre-requisite amount work, yet only a tiny amount of work is needed on the validating side.

This “work” therefore equates to time.  By varying the probability of finding the value with a qualifying hash you can adjust the difficulty of the puzzle dynamically so that, on average, the qualifying hash can only be found once every 10 mins.  The nature of the puzzle makes it easy to add difficulty and simply negates progress in compute power solving puzzles ever faster, which would otherwise top out the max coin in circulation in seconds.  Therefore a block (and ultimately new coin) is added to the chain once every 10 minutes, limiting the amount of currency available.  

The tick of the blockchain clock moves forward one block at a time, every 10 minutes, and this clever cryptographic chaining of transactions, combined with incentive based mining both validates the ledger, adds to the currency, and provides the distributed clock.  Clever and powerful stuff.

Nb. A timestamp is accepted as valid if it is greater than the median timestamp of previous 11 blocks, and less than the network-adjusted time + 2 hours.

So when the blockchain paper refers to a timestamp, it is talking about linking a cryptographic chain with a puzzle that acts as a timestamp based on distributed consensus, where everyone agrees on the timing of the ticking of the clock (and therefore the sequential order of the blocks in the chain) by how hard it is to solve a puzzle.  To labour the point, the difficulty of finding a qualifying or conforming hash acts as the clock.

It doesn’t matter that this clock is not nanosecond precise, just that everyone agrees with the timing.  The clock is universal, meaning the difficulty of the puzzle is universally hard enough that no matter who it is, you have to do a certain amount of work to solve it, CPU cycles etc.  When it takes around 10 minutes for everyone working on it to typically find the magic answer and win, everyone agrees.

Technically the clock is dictated by multi-exahash rate of an unknown number of completely independent but collective participants across the planet.

Nb. There is also 51% problem whereby if someone can control over half of the compute power in the world working on the puzzle they are much more likely to win every time, and in turn cheat the system.  The system, however, also has a canny way of highlighting this as an anomaly and shutting it down, but maybe that detail is for another time.

It has taken a while for us to get here, but with all these components making up a distributed and validated time-stamp linked to transactions, we have ourselves a crypto-currency.  Let the games begin.

Caveat emptor

Hype train

It should be noted that for all it’s potential it is still very early days for cryptocurrency and blockchains.  A good deal of research needs to be, and is being done to bottom this out.  Barely a week goes by without yet another large tech headline around a potential use-case or product (this week’s was VMware around Enterprise blockchain services).  There are adverts on television with talking washing machines proudly boasting blockchain capabilities. The term is firmly becoming part of the lexicon, if often incorrectly used and massively over-hyped.

Hopefully this article gives a good enough starting point to see through some of the hype and evaluate use-cases more realistically.

Recent events show how this hype, over-exposure, and acceptance of terminology which most don’t really understand, can gloss over any potential pitfalls so they no longer seem news-worthy.  Bithumb, South Korea’s largest crypto-currency exchange, lost $30 million to hackers, and it barely hit any headlines, even in Tech news.  In addition to this, Bitcoin, Ripple, Ethereum and Litecoin only dropped around 2% in the immediate aftermath, which is significant but hardly a crippling loss of confidence.

There is also a rash of speculation over Initial Coin Offerings (ICOs), which are similar in concept to IPOs  (Initial Public Offering – a process by which companies can raise capital), but instead of shares, the investor gets tokens or coin in return for investment.  Much like a normal investment but without the pesky regulation providing safeguards for your money.  Remember if a new coin offering introduces a new crypto-coin (i.e. a brand-new crypto-currency – let’s call it NebulousCoin), there is no guarantee that this will be worth anything in future.  The value is entirely based on market hype and speculation.  Delve into some of these ICOs and they are literally thin air  – a few words for planned product, much hype and no substance.   My advice would be to do some serious homework before thinking of investing.  China has in fact banned ICOs, and although it goes against the ethos of crypto-currency Utopia, more and more noise is being made around regulation being a necessary evil if ICOs are to survive?

ico_category
Source: Coinschedule.com

 

Then there are the security concerns.  Crypto-currencies turbo-charged the Ransomware industry.  Legitimate mining seems to be on the decline as hackers target companies to take advantage of their army of CPU resources to win more often.  The real security issue of hacking for mining rewards with an army of compute makes it hard to compete with a garage computing rig.  The odds and costs look less and less attractive.  Then there is the risk that after all this hard work your system is compromised and your wallet hacked and emptied.  Taking advantage of a software vulnerability is much lower hanging fruit than targeting Crypto.  As ever, where there is money there is opportunity for nefarious profitable activities.  The bigger the reward, the bigger the incentive, and  crazy market valuations only add to this problem.

To sum up..

In this article we have looked at Currency in general and how it relates to crypto-currencies like Bitcoin.  We have looked at what Blockchain is and how it underpins the system.  We have looked at how blocks are added, the concepts of hashing, how transactions are validated, and how this leads to a digital currency based on distributed consensus with timestamps as the ticking of a distributed clock through proof of work.  It is fair to say we have covered a good deal.

Hopefully from this you can start to see other applications of blockchain technology, particularly around linked validated records of information and secure distributed databases.  Equally I hope you have noted that although the concepts are clever,  there is a need for caution, understanding, and research, before blanket acceptance of anything associated with “blockchain” is seen as inherently good or ‘better’.   There is great power here but, as a well known super-hero says, “with great power comes great responsibility”.

So there you have it, crypto-currencies laid bare.  Who knows, the next time you hear someone throwing the term “blockchain” around to save the world, you might just start thinking…

 

Advertisements

The Tech – Finance divide

Take two technical areas, put them side by side, make sure whichever side you are on you make no effort to understand the other, while assuming your area holds the keys to the kingdom.  If only the other understood your particular discipline they would have a moment of enlightenment, realise the folly of their endeavors, and worship a deeper truth.  After all, your discipline is operating on a higher evolutionary plane.  This can be the case between technology and financial areas within an organisation.

The bridge between these two technically skilled departments is often so imperfect that you can’t even see the other side, just told there is a mystical land where things are done very differently, but please do provide the following presentations, customer data and figures needed to cross the chasm.  Often the bridge is built by other departments (Sales, Product, Marketing etc.) with their ‘unique’ insights to the customer, market, and the perceived best needs of the organisation.  Let’s be clear there are no ‘unique’ insights that cannot be explained to all departments with a little patience.  All departments have field of vision weaknesses.

On the technology side, I have seen leaders so impressed by the potential of a certain transient feature or widget, they put it at the centre of revolutionizing an entire organisation.  I  have also seen sales leaders confidently declare that no-one remotely cares about technology in the slightest, even though customers are purchasing precisely that technology to solve their business problems, and ultimately drive the sales number.  (Remember during the financial crisis.  One of the well publicised problems was senior management not really understanding the products they sold.  Thank goodness this doesn’t happen in technology).

On the Finance side, I have worked in, and with, organisations where the chief execs have struggled to see what value Finance actual brings to an organisation other than keeping the regulators off their backs and balancing the books.  Yet others have moved the entire focus of the company to being a Finance house on the back of temporary interest  imbalances, exchange rate advantages across geographies, or the ability to raise capital finance for customers.

What I am saying is that no single area is perfect, and although all these departments undoubtedly play key roles, the extent to which any single one should dominate the direction of a company is open to debate.

With such varied opinions it only makes it more important to foster understanding and break down silos.  No one department holds the key to organisational or business wisdom, this comes as a collective.

To this end I thought I would do a little bridge building for technologists to reach out to their colleagues in Finance with whom they have much in common, but may have ignored in their quest for technical mastery.  As below, the bridge between departments, knowledge, and your own viewpoint is rarely a straight line, but we can try to plot a path.

 

I have chosen just a few terms and concepts that people run into frequently when getting technology business cases approved and projects off the ground in order to 1) break down silos   2) show how easy some of this is conceptually, and  3) gain confidence in initiating conversations, jump into further learning, and help make a case for your technology revolution.

Let’s tool up and dig into a few business case basics.

Net Present Value

First, let’s take Net Present Value.   Why?  Well it is used in many places and is the basis for many a project decision (the spreadsheets you may have to fill out blindly in order to get the “highers” to approve your project).

Ultimately what we are looking for is a way to make a financial decision on the viability of a product or project, usually an accept or deny decision (go / no go) on a project.  Net Present Value is also a tool we can use to compare projects (with appropriate caveats).

The equation looks a little scary, and enough for some technologists to throw up their hands and say  “What’s this?  These people just don’t get it, can’t they see this will make us a fortune?”.  Bear with it though, as it is not as crazy or complicated as first glance may imply.

In short if the NPV or Net Present Value is greater than 0 then accept.  If not, deny.

Let’s start with the formula as a base and go from there.  It’s not that difficult, so hang on for the conceptual explanation.

The formula for NPV is as follows.

F44305E2-940A-40C0-AF84-17FF29B508BA

Where:

Ct = net cash inflow during the period t

C= total initial investment costs

r = discount rate, and

t = number of time periods

The ‘Sigma’ or ‘Sum’ symbol at the beginning of this equation is just saying sum up the result of the following, and t is a time period.  So if you had 5 time periods (or say 5 years), you would just sum up this result for each year to get the final NPV.

It is just a shorthand way of saying the following:

NPV 1

Now for some explanation:

If you have the option of money in your hand now rather than the same amount at a future date, which would you choose?    You know what the money is worth today.  There is no risk and the value is what it is as it lands in your pocket.  It is potentially worth more now because you can use it within a business to buy something today and sell for profit later, or to invest in products to be sold with the added value of the sum of the parts, to make more money relative to the initial outlay.  If, however, we wait a year for the same amount of money then there is risk associated with it.  It could be put in an interest account to offset a decrease in value, but there is always the risk it will be worth less due to inflation.  This is called the time value of money.

Close Up Of Time And Money With Green Bokeh Background ,business

Time value of money is useful as it allows us to compare projects that do not necessarily have the same investment or revenues, or see the cash flow at the same time.

Say I have 2 projects. The first I invest £100k with projected return on investment in 12 months of  £120k.  The second I invest £130k, but I have a projected return of £160k but in 18 months  time.  I can’t simply compare the net return of project 1 (£20k) with the net of project 2 (£30k), as time makes it a poor comparison.  What we need to do is work out what the £20k and £30k would be worth today – the present value.  We do that with the above formula.

The formula takes into account a few factors:

1) the cost of the initial investment (C)

2) the money the initial investment will make over a period, or the net cash inflow (C)

We then take into account

3) (r)   the discount rate – which is the rate at which we have calculated the value of the money vs today’s value will decrease over time (say 5% a month).

and finally

4)  (t)  the number of time periods.

So what we are essentially saying is we have a number of time periods, (months for example).  For each month we work out how much money we think will come in for that month from the project (or investment), get a figure, and then work out how much that would be worth if the value of money has gone down over time (the discount rate – as a representation of the time value of money)

Let’s take my example above and plug the numbers into the formula with a discount rate (r) of 8% over a single time period (12 months).

NPV1

so with “r” at 8% you get..

NPV2

NPV  = £11.11k

NPV is greater than (>) 0 so the project more than breaks even.  Good to go.

The time period can be anything really, it could be per year, or smaller periods that contribute to working out a much longer period.  In this case project 1 is projected for a year, and discount rate is 8% for the time period, which makes the net present value of project 1 not £20k but around £11k (£120K return which in 12 months time is worth £111k, minus the initial £100K investment).

Project 2 however has a longer time period to contend with, and a 14% discount rate for the period, so although it looks like project 2 will make more on the surface, after NPV is taken into account it actually only makes £10.4k, so less than project 1.  Added to that you have an extra 6 months on the timeline.  We are making estimates into the future around costs and returns, so it stands to reason that the margin for error, and in turn risk, may increase.

As you can guess, Net Present Value has it’s limitations and depends largely on the quality and accuracy of the data plugged into it.  Often it requires a number of estimation techniques which, at best, give a good indication of whether something is worth doing, or at worst are tantamount to soothsaying.

Let’s have a look at 3 significant caveats.

  1. Calculating cash flow.  I went into this in a little detail on a previous post about EBITDA, but essentially there are 3 broad sections on a statement of cash flow to look at.  First is operating cash flow or cash acquired through normal business operations  – things like selling products, providing services, paying salaries, selling and purchasing inventory etc.  Second cash flow from investments, so money spent on capital expenditures and fixed assets.  it also includes buying and selling stocks.  Things like equipment and real estate. Thirdly cash from financial activities, like borrowing activities, or loans from banks, money paid or received from loans etc.  Cash increases when liabilities are taken on and decreases when paid off. It is important you understand the effect on these areas when putting a “cash flow” figure together for your business case.
  2. Estimating how much a project or expenditure will cost. A static piece of equipment with a fixed point in time price holds very little risk, but time and resources, 3rd party help, timelines etc. can be difficult to calculate accurately.
  3. Calculating the discount rate.  This is a forward projection and the discount rate today may not be the discount rate tomorrow or in a year’s time, which could mean costs go up (interest rates etc.).

Not only are there the caveats above, there is also more than one way of calculating whether something is worth doing.

NPV is one method, and IRR or Internal Rate of Return based on NPV is another.  Let’s take a quick look..

Internal Rate of Return

IRR  = The Internal Rate of Return, which is the interest rate that makes the Net Present Value equal to Zero.  That make sense?  Maybe not.  What it actually tells you is the speed at which money comes back to you, and is written as a % per year.   What this means is playing a bit of a guessing game.  You have your discount rate of say 8% in the NPV formula and you end up with a number e.g.  NPV = £2000.  You then try to find (or guess) an interest rate that balances an equation to make NPV = 0.  Say that ends up at 20% interest (you are essentially balancing the equation to make the NPV = 0).   If the interest rates is greater than the discount rate, you are making money.   To think of this another way, the IRR is the discount rate that makes the NPV = 0, and when the NPV = Zero you are at break even point.   Therefore if the IRR is greater than the discount rate you used in your initial NPV calculation, then you are going to make some money (or value) over time.

Let’s take our previous example

In the case of IRR we are trying to find an interest rate that makes the NPV = 0, so for this we use a big “R” as the interest rate, then we take a few guesses (keep the discount rate (r) to one side for the moment and remember it was 8%).

IRR1

In this case you could guess say 15%, but this makes the NPV = 4.3.  Eventually you guess 20% for the interest rate (i.e. 20% on top of your initial outlay or loan is returned – the interest).

IRR2

NPV =0

Finally NPV now equals 0

Once the NPV is 0 you have your break even Interest Rate, or IRR.

Therefore if the discount rate used in your NPV calculation is less than the IRR you will make money.  The higher the IRR in comparison to the discount rate the better, but as long as it is greater than the discount and the “hurdle rate” for the company for new projects, then this should be a tick in the accept box.

In this case the Interest rate is 20% to make the NPV = 0 and the Discount rate is 8%.  Therefore the IRR is greater and the project – a tick in the accept box.

IRR does have limitations which do not take into account the size of the project.  For example, if you had a project that needed £1 invested to return £3, and one that you invest £1m to return £2m, as IRR is a percentage you may conclude that the first project is much more attractive even though you make much more money on the second.

Also when calculating returns it is important to understand how funds are raised to invest in whatever you want to push through, i.e. the cost of capital, as this will influence the discount rate.  Usually this is the Weighted Average Cost of Capital (WACC), which is basically a weighted average of debt and equity company wide. For example if you are funding something through debt at an interest rate of 10%, then the capital costs 10%.  Also it does no harm to be aware of the Opportunity Cost of Capital (OCC) where there is existing money ready to go to fund a project.  You are then looking at the cost of not using this money elsewhere on another investment or project, i.e. the cost of foregoing another opportunity.

Money tap

There are many other things to consider here.  Indeed just defining what everybody means by cash flow can be a challenge (Free Cash Flow, Discounted Cash Flow, levered and unlevered cash flow, cash flow from operating activities, cash flow from financing activities etc.).    Regardless of the finer points, a basic understanding of the above will serve you well.

Hurdle Rate

By understanding the above you can get a better view of a company’s “hurdle rate” which is the minimum rate a company or controller expects to earn when investing in a product.  Generally if the IRR is equal to or greater than the hurdle rate (also known as the minimum acceptable rate of return – MARR), projects are likely to be approved (company politics aside).  How does a company arrive at a hurdle rate figure?  It is usually related to the cost of capital for the company, which, generally speaking, is the costs of funds used to finance a business (cost of equity and cost of debt).

Some useful tools then.

However, as with all things where projected costs and returns are involved, there is always uncertainty, and a real risk of “garbage in garbage out”  You can have all the tried and tested formula in the world and on the surface it seems like there is enough for a go / no go decision, but if the data has been put together poorly with little market insight or technological understanding, the figures that pop out are worth little more than saying “we think this is a good thing, do you agree?”.  This is usually where company politics or stakeholder buy-in plays an out-sized role.

Ambition also plays a large role in many organisations with less formal or tested processes around requesting investment.  Some will say they can sell 6 of a widget in a quarter for fear of missing projections, lack of market knowledge or sales force buy-in.  Equally someone else in the organisation will be convinced they have done their homework, market research, got stakeholder buy-in, and are equipped with a highly motivated sales force, so believe they can sell 100 of their widget.  Insert the ambitious figures into the methods outlined above and guess who normally wins?

What is also true is that many companies are not terribly good at reflection, so a mistake at this stage is rarely highlighted later when everyone is focused on the next big thing, or steadying the ship.  Most requests are put forward with the best of intentions and beliefs, so given markets are liable to move in and out of your favour, it is hard to predict a sure thing, and doubly hard to look back with accountability.  The key is to build a track record of success.

Finally, as Technologists love learning new skills, my point is that it does no harm to understand and speak the language of the business case spreadsheet when trying to get things moving.  Not allowing people to hide behind terminology, organisational structure, or even cult of personality, can only be a good thing for constructive, transparent collaboration.

So next time you are dealing with a financial controller at arms length, passing back and forth spreadsheets, stop!  Pick up the phone, invite them for a coffee, and ask if they can spare some time to explain their world to you (and yours to them).  Most people love explaining what they do and the challenges they face.  Sometimes the issues we are dealing with can be strikingly similar, and only made worse by retreating to our Islands when no-one appears to be listening.

 

 

 

 

The Product, Service, Solution Triumvirate

banner_integrated-solution

Something often forgotten, is that demand for, and sales of, products or services come as a result of solving a problem.  This is generally true across the board and particularly with business problems.  This is also why use-cases demonstrating what is being solved to support business outcomes, are so key to product, service, and solution development (e.g. make things easier, reduce costs, save time – time is money and competitive advantage).  In agile development the concept of user stories are powerful precisely because they relate to the above.  That is, they describe the type of user, what they want, and why  (For a well described type of user e.g. Admin… i want to do something… so that I can…achieve something).

To this end companies create products and services to enable solutions to business problems.  All too often, however, the distinctions between the three are blurred, confused, or simply lost.  The result? A confused workforce working all hours with focus on component parts and not the wider customer fit, creating something they don’t fully understand and confusing the customer.  It is no wonder marketing can get the wrong end of the stick when trying to get the message out, sometimes only very loosely reflecting reality.  It can look and sound good of course, but the pain inevitably comes later when customer expectations are not met and so many people have been involved that it hides the answer to “where did it all go wrong?”.

Why the confusion?   The general business directive nowadays is to sell “solutions not products”, so teams are very keen to make all products sound like solutions.  Instant added value via a creative narrative.  The problem is that it doesn’t necessarily make propositions any clearer for customers, nor is it clear that it adds any value.  Blurring the definitions is tempting, but not altogether that helpful .

To that end, let’s have a quick refresher around the basic triumvirate.

Product – a product is essentially a trade-able item, a thing, or some stuff  (be it physical or virtual).  Something that is created and can be exchanged for a fixed cost.  Products are heavily associated with value, (value for money, economic value, benefit to economic agents, market value etc.  but let’s not get into an Economics 101 here).  Confusion sometimes comes from the fact a product can be a good or service which relates to a unique offering.  For clarity though, this typically will have gone through a formal development process (defined feature set, naming, pricing, delivery models, documentation etc.) to produce a “product”.  Importantly, although products can provide many features and benefits, that does not make it a solution!

Service  Often, having a well defined and developed product is not enough in and of itself, and requires services to release the full value potential of the product.  By applying knowledge or skill-sets, the work of services is then delivered by actual people to other parties.  In this sense a service can be delivered independently of a product, (a professional service like maintenance, implementation, repair, or sharing of intellectual property e.g. design expertise etc.), but often they are used to implement and support products.  This is particularly the case with complex technology products where a level of understanding is needed to get the most out of the product.

A service is not trade-able, or something that can change hands, and in that sense is not fungible.  If a service truly becomes like this, then it can be defined as a product.  You can see where confusion can start to creep in.

So essentially a service is work done by a person for another party, and as such tends to be quite people and skill intensive.

Solution:

Product service

·

Solutions are a slightly different beast with different skill-sets.  The idea being to bring products and services together, and developing a system to integrate the two into an end-to-end solution.  Solutions look at customer requirements and reach into the tool-box of products and services to produce a solution that best satisfies the customer’s needs.  Ultimately it must deliver a positive business outcome, which is why understanding the problems of customers and their desired business outcomes is essential when putting together an effective solution and adding value (e.g Drive revenues. Reduce expenses. Increase Margin. Improve efficiency, eliminate waste [Time, Inventory, Motion]. Drive customer satisfaction. Mitigate risk etc.)  Essentially building the value proposition of the customer solution.

The back-end of this (often forgotten in a “sell and forget” world) is measurement of results and proving an effective solution has been delivered.  It is a combination of product and services to this end.

Where to from here?

With these basic definitions it should be easier to start asking questions like.. “Can the service be a product?”,  “Can the service be a solution?”,  “Can the solution be bundled up as a product?”.   By understanding that ‘products, plus services, make up solutions’, and with feedback shared and understood across teams aligning to business objectives, product owners can be put in place with the confidence that what they produce will hit the sweet spot in form, functionality and quality.

.

 

I am personally a fan of a tangible product owner, much like the chief engineer in Lean Production, who plays a key role in the product development system.  A highly respected, senior engineer appointed by, and reporting to, senior management.  This role combines the 3 traditional roles of Product manager, Project manager and Chief Architect.  The role is a bit like a Superman Lego Architect that builds a Lego Jet Engine.  A combination of  technical, architectural and customer facing skills, with leadership and commercial expertise is a must.  This is no mean feat!  Development of this highly respected skill-set takes time and effort of course.  Lean systems have a policy of rotating key contributors around areas of the business to grow their leaders long term, and rely less on networking or personal branding.  You don’t become a doctor because you spent some of your time studying ankle joints.  You need to understand complex biological systems as a whole in order to assess situations.  Based on understanding, good doctors can work out the best treatment and outcomes for patients.  By using products, skills and services they provide… a solution.  This system-wide understanding, also enables robust flexible services to be built  and continually improved, with responsive feedback into the product life-cycle.  Yet more positives for customers.

Back to the definitions and how to be clearer internally and externally?  We really should try to avoid “solution washing” where possible, as everything ends up looking the same and nothing distinguishes itself.  If you understand what you are offering, you have done your research, know your value proposition and use-cases, and what problems you are solving  – go with it!  If you are using responsive agile methodologies you will also be able to adjust as you go and hit the mark.

If companies speak simply and directly to a need, help customers understand the claims they are making and why, then, by knowing the difference between products, services and solutions, they should be able provide real value without hundreds of mice scrabbling in the background working endless hours to customise something, only to look vaguely like it should have done in the first place.

Taking the time to understand and be clear on the triumvirate is crucial to develop, own, collaborate and communicate offerings properly.  Get this right and your customers, employees, colleagues, and shareholders will thank you, trust me!

Customer-Satisfaction-700x467

 

DDOS approaches and Flowspec

Dos imge

 

Denial of Service – for once a technical term that is pretty much exactly what it says on the tin.  Essentially the service you are trying to provide (be that internet access, a corporate application, connectivity), is denied.  I unplug your TV and you can no longer watch TV, denial of service, simple.

In a volumetric Denial Of Service, where a large “volume” of traffic is created to fill up your connectivity pipe, typically you need some way of creating the traffic and sending it towards your victim.  This could be a single machine pumping out traffic, but this is rather hard to scale nowadays for the amount of traffic needed to clog some pretty beefy pipes.

Distributed Denial of Services, again, means what it says.  Instead of using one machine, you use a lot of distributed machines to all generate traffic simultaneously and point it at your victim (or many victims) to deny a service.

coffee

What is the problem?

Denial of Service and Distributed Denial of Service have been around for some time, but with the advent of IOT (Internet of Things) and proliferation of cheap devices with basic distros and firmware, the scale of DDOS has exploded over the last year or so.  The major headlines making people take notice were around the Dyn DNS attacks – which were a combination of the Mirai botnet and Bashlite, affecting the services of Twitter, Github, Dyn DNS, Netflix, Reddit etc.  Variants thereafter affected many Deutsche Telecom and Talk Talk routers to name a few.

This, in turn,  signaled the dawn of Terabit per second DDOS attacks with tens of millions of IP addresses.

ddos

Mirai and Bashlite

The Mirai botnet scans Ip addresses of vulnerable IOT devices with default usernames and password, reports to a control server, and from there can become a BOT to be used for DDOS.  It also removes competing malware from memory and blocks remote admin ports.  The source code for Mirai was dropped into the open, so effectively became open source.

Mirai Nikki
Mirai Nikki (Future diary)

Bashlite came out of a flaw in the Bash Shell (ShellShock) which took advantage of devices running BusyBox.  BusyBox is software to provide unix tools in a single binary – originally for Debian distro, so instead of each app having its own binary, you can call the single BusyBox binary with different names for various apps.  Essentially this allows code to be shared between multiple applications without requiring a library to reduce overhead.  Useful for memory constrained, portable operating system environments – POSIX – hence IOT).

From booters and stressers, to unpatched NTP or DNS internet services,  old home routers, 4g/5g handsets, all variety of IOT devices with basic firmware, flimsy linux distros with secondary passwords etc.  if these types of devices are converted to a network of botnets, then you can ask them all to send traffic to somewhere and the traffic scales rather quickly.    A brief Internet search for booters and stressers reveals just how easy it is to rent a service to cause annoyance and disruption (not that you would generally want to do that of course, as they are mostly illegal for use in the wild, but for testing purposes for your own infrastructure with correct permissions … maybe).

Unfortunately in a rush to get basic functionality out there, the market has been flooded with a variety of cheap IOT devices (or IOCT  – Internet of cheap or crappy things).

As many of these cheap IOT devices never get upgraded through hardware, firmware or software, or swapped out all too often, we shouldn’t expect this problem to go away any time soon. (Mirai, incidentally,  means “future” in Japanese.)

This is the general problem, so where typically on your infrastructure does it present?

The Denial of Service problem typically affects several places in the network, most commonly the Internet Pipe, the Firewall and the actual server under attack, but SQL servers, load-balancers, IPS/IDS are also affected.  The failure of Firewall, IDS/IPS or Application delivery controllers to address the problem (they tend to get hosed in a serious attack), has lead to dedicated DDOS solutions in the market.  As Internet pipes tend to be hit the most a lot of focus has ended up there.

So what is the solution, and is there a 100% effective solution today?  (I am guessing we already know the answer).

Historically the first way to mitigate an attack would be for an enterprise who is being DOS’d to contact their friendly Service Provider upon detection and ask for help. The SP would then perhaps install an ACL or provide filtering ad-hoc.  This could be a short conversation and ultimately money based – so not altogether that practical long term.

A stellar-mass black hole in orbit with a companion star located about 6,000 light years from Earth.

Another way would be to use DRTBH (Destination-based remote-triggered black hole), which essentially needs SP acceptance of BGP advertisements from the Enterprise as part of an agreement.  The Enterprise, by means of a BGP community attribute, would (upon detection of an attack), advertise say a /32 to the SP and then this would be blocked.  Of course this effectively completes the DOS for the host but could be of use for investigation and damage limitation.

A second method following this is to use SRTBH (Source-based remote-triggered black hole) which again involved BGP interaction, but this time the SP would look at blocking the source of the attack, typically in relation to something like uRPF (unicast Reverse Path Forwarding).  Remember BCP 38?

There are two forms of uRPF – Strict mode and Loose mode.  In strict mode the packet must be received on the interface that the router would use to forward the return packet. This has the potential to drop legitimate traffic if you have asymmetric routing for example, i.e. say you receive traffic on a interface that is not the choice of the router to forward return traffic.

In loose mode, the source address must appear in the routing table, not necessarily the actual interface, and allows default routes.  You can also configure filters to permit or deny certain sources.  There is also an option for the ISP-to-ISP edge (uRPF-VRF) to allow uRPF to query the VRF table containing all routes for a specific eBGP peering session over the interface, verifying the source addresses of packets matching the advertised routes from the peered ISP.

Not so much DDOS mitigation as a fairly manual or prescriptive dropping / blackholing of traffic really.  I tend to see this as damage limitation.

—-

Of course if you can identify an attack manually or by notifying your service provider then you don’t actually have to drop the traffic.  You can redirect to a scrubbing or cleaning service, e.g. maybe a manual PBR redirect to drop the traffic onto a VPN/VRF, but generally it is better if there is a method to inform and push policy to redirect real-time, based on an attack in progress.

To this end we now consider BGP Flowspec, which is a little like enhanced PBR (Policy Based Routing) for more granular policy decision making and policy distribution across the infrastructure.

—-

BGP Flowpsec

If you are going to drop traffic or at least differentiate between what you think is bad traffic and have some granularity of control, then BGP Flowpsec becomes a good option. If you know the type of traffic you should not be seeing, you can do worse than Flowspec to both identify traffic and distribute policy to the routers in your network that need to drop the traffic.  Indeed, based on policy, you can also determine which traffic you need to redirect and then redirect to a dedicated DDOS scrubbing device or service where “clean” traffic is returned to the user and service continues despite an attack taking place.

So what does BGP flowspec do? Policies

Well you can effectively create a policy to match a particular flow with source AND destination, and L4 parameters with packet specifics such as length, fragment etc, and allow for a dynamic installation of an action at the border routers.

If this policy is matched then you can perform an action:

  • drop the traffic
  • Redirect the traffic – e.g. inject it into a different vrf (for analysis)
  • or allow it, but police it at a specific defined rate.

Much like rate-limiting polices and QOS but with specific malicious flows that you recognise.

BGP  Flowspec basically adds a new NLRI into BGP (AFI=1, SAFI=133), NLRI is a field in BGP that, at its simplest, is used to identify the prefix for BGP advertisements (literally Network Layer Reachability Information), but as a variable field, you can use NLRIs in BGP to represent pretty much anything you wish (BGP is being overloaded with all sorts nowadays with this NLRI field, think EVPN etc.).

In this case you add information about a flow as below :

1.Destination IP Address

2.Source IP Address

3.IP Protocol

4.Port

5.Destination port

6.Source Port

7.ICMP Type

8.ICMP Code

9.TCP Flags

10.Packet length

11.DSCP

12.Fragment

10.Packet length

11.DSCP

12.Fragment

 

Now you can match based on the above and define what characteristics of traffic are most likely DDOS. Indeed it is typical to have a Netflow feed off to an analysis engine to determine which traffic needs cleaning or is DDOS traffic and then use BGP Flowspec to instruct the network which precise flows to redirect to a cleaning service based on specific defined parameters as per the above – real time! (redirect/next hop modification, DSCP remark, drop or police, VRF leaking).

BGP flow-spec is a client-server model, so you can have the analyser as the server dictating to the client what you would like to match and action on, and then instruct the client (router for example) what to drop, police or redirect.

Flowspec is therefore a useful tool for policy distribution, drilling into specific actions on specific flows with traffic characteristics, and as a method to inform redirection, rate-limiting or dropping, real-time. Now let’s have a look at some typical types of DDOS attack.

Types of attack

Amplification attacks (dns, ntp, ssdp, snmp, chargen, qotd etc.).  The idea here is that traffic is spoofed.  By not using a full handshake, a large answer is sent to the victim’s address, or takes advantage of vulnerable protocols on large servers.  DNS is a prime example, where DNS responses can be much larger than the initial request – up to 4096 bytes with EDNS.

An example mitigation for this is using rate limiters for traffic and ports that have no business crossing network boundaries – SSDP UDP 1900, Netbios UDP 138, NTP 123, Chargen UDP 19 (character generation stream, not that prevalent, but there have been cases), fragments and large TCP Syn packets.

On some platforms you can be even more granular.  Instead of rate-limiting per class of traffic per interface, you can rate-limit per user (micro-flow policing) e.g. for DNS and NTP – if you see excessive traffic of this type then it is likely an amplification attack. Understanding your traffic patterns as best you can is of course key with these techniques.

At a Service Provider level, many of these amplification attacks can be blocked with router config.  There is no desperate need to send off to a cleaner, as routers can do a quicker job without the redirect if patterns are identified at the edge router with ACLs or BGP Flow-spec (maybe informed by an analyser).  You do however need to be extremely careful and know exactly what you are doing.

At the Enterprise level it is usually a bit late by the time the attack reaches you, so BGP flow-spec or cloud services might suit best.

It also makes sense to implement a bunch of security best practices on your infrastructure itself   e.g.  Generalized TTL Security Mechanism (GTSM) for  TTL Security   (RFC 5082 if you are interested.) Protect your control plane and management plane on devices through policing and protections, , MD5 auth for your routing protocols, key chaining, using SSH/SFTP/SCP and of course AAA where possible.

Layer 3/4 stateless volumetric attacks (udp frag, icmp flood ) usually filtered at the edge router typical of an SP or inline device for on premise (DDOS inline box).  If using on-premise or inline (e.g. scrubbing device inline, maybe on a s firewall), then if the pipe is already hosed before you get a chance on prem, seeking help from your service provider is a priority.

datascrubbing

Stateful volumetric attacks – tcp syn, http, ssl, sip  (e.g look deep into Syn packets for legitimate replies)  For these attacks you need a more intelligent scrubbing device, and ideally deploy as close to the edge as possible.  These attacks are typically scrubbed at the PE or SP data centre (like an SP with one of the leading DDOS applicance vendors).  This is locally hosted in the provider and gets around the tricky routing with BGP external and returned scrubbed traffic using  GRE tunnels –  much easier when it is your own infrastructure and routing.  Another option would be cloud scrubbing of traffic and clean traffic returned.  If you are tackling this on premise then maybe you can collapse this function into a FW or ddos box., as long as the DDOS inspection and mitigation is done first before other security or traffic controls.

Slow-loris
Slow Loris

Finally your slow pace attacks that can be stopped in the cloud or require an inline solution (slowloris slow and low, http flood, ssl floods, sql injections, xss csrf, app misuse, brute force, server resource hogs etc.) where an SP doesn’t typically have much visibility of the types of attacks that slowly exhaust the resources on the server.  For example Slowloris where sending HTTP headers in tiny chunks as slow as possible and waiting to send the next chunk until just before the server times out the request.  The server then is forced to wait for the headers to arrive – so if you quickly open enough connections, the server struggles to handle legitimate requests.  Or R.U.D.Y (R U dead yet?) where you send HTTP POST requests with an abnormally long ‘content-length’ header field and then start injecting the form with information, one byte-sized packet at a time, every 10 seconds (or random intervals in some ddos protection evasion techniques), with the long content header stopping the connection being closed and exhausting the connection table.

You should consider enhanced reporting on these attacks.  Netflow is essentially a sampling technology, so in order to spot these type of attacks you might need an inline device. They need some pretty deep inspection of each packet and anomalies so it is nigh on impossible to do this effectively at several 100 Gbps.  Keep this in mind when paying for cloud solutions and any premiums.  The rule in general is to do this as close to the resources you would like to protect as possible, hence inline, and understand the performance limitations.

Types of redirection

DNS redirection to the cloud is one method, and very popular at the moment (everyone loves clouds in IT but, oddly, no-one expects rain).  With DNS redirection, when you resolve to an IP address, you actually resolve to the DDOS protection service address and therefore go through the scrubbing protection before being served.  This can be on-demand or always on.

A second method is BGP based “inter-as” DDOS protection. Similar to BGP hijacking. Effectively your Service provider advertises a more specific /24 block for your site address (e.g. ip address 1.2..3.4/24) and as this is more specific routed traffic is magnetically drawn to this advertisement, which in turn redirects you to a scrubbing site first, before returning clean traffic back to you over a GRE tunnel. You need this tunnel of course to prevent a routing loop – you send traffic back to the original address, which again gets picked up by the /24 and ends up back at the scrubber for ever and ever. It is also possible to tie up a /24 permanently to specifically cater for DDOS.  One other method that some providers are using is to effectively act as an IP proxy – give away their own public IP addresses dedicated to you to obfuscate your own in an “always on” type service (for l3/l4 volumetric), bit like dns redirection, but a new IP advertised that belongs to the cloud.  When you get the traffic back you need to NAT etc. for your own range.  The caveat here is that it is always on, and all your traffic goes through the cloud provider first.

Typically the tunnel is what you are paying for from the cloud provider.  Remember, if you are doing a /24 BGP advertisement, you are essentially doing BGP hijacking. BGP origin validation could make this approach more difficult in future.

There are various flavours here – your edge router can act as a detector sending sflow or Netflow to your cloud provider, who, upon detection of DDOS, takes over the routing via BGP advertisement to redirect to the Cloud DDOS provider for the duration of the attack – on demand service (GRE back to the edge router for scrubbed traffic), or a permanent IP advertisement so everything goes to the cloud by default – always on service.

Service providers can of course bypass this and provide a hosted DDOS solution, where all the redirection happens internal to the service provider in a local SP Data Centre where they scrub the traffic for your Internet connectivity.  As above, this can be a premium “always on” service, or a service that kicks in under attack notification either automatically based on suspicious detected traffic or on request from the customer.

Finally let’s have a brief look at places in the network

Places in the network

Once you understand the type of attack you are protecting, then you can look at which service and place in the network is most effective for each, and whether you have covered what you need.  To be honest, whenever most people talk about DDOS they jump to volumetric – my pipe gets filled, re-direct and clean please.

To summarise some of what I have mentioned above, you have a bunch of options:

“In the cloud” services from a DDOS vendor where traffic is redirected either permanently or upon detection of an attack, cleaned, and returned ( Your method of redirection is either DNS based or BGP inter-as based).  An ISP hosted DDOS service; your SP can redirect you to a cloud vendor DDOS service, or stand up their own detectors and scrubbing service in their own Data Centres to monitor traffic at their Internet peering points and potentially provide a service back to a customer – like a “Clean Pipes” solution. Or you can go On Prem (centralised, distributed, mixed, inline).  Finally you can of course mix and match these approaches to make sure you cover as many conceivable denial of service attacks as possible at a latency that suits your applications, even down to deep packet inspection.  I guess there is no one size fits all.  One thing to note is if you use a Cloud provided DDOS, then to protect web-sites/L7 a proxy based DNS service usually works for most stuff and they maybe look at the l3/l4 attack prevention on demand should the attack circumvent the DNS and head straight for the real IP.  In truth multi-terabit, deep L7, slow and low cleaning etc. at any kind of reasonable latency and cost doesn’t exist today, which is why you protect per asset or web address or as a percentage of traffic.

Ultimately in future I expect DDOS protection to be a given as part of any Internet service consumed (it mostly already is), and premiums for this kind of service will likely come down, depending on volume of attacks, complexity and scale.  Cloudflare, for example, have just announced that DDOS is bundled at any scale within their Internet services as part of the usual rate.

New vectors tend to mean new solutions to problems so I would expect this to be charged for, wherever significant investment has been needed to solve the problem.

So there you have it, a brief tour of DDOS approaches being used in the market today. Implement the above, scrub yourself down, redirect yourself to the nearest good cup of coffee, and relax knowing your traffic flows gloriously uninterrupted…for now.

shiny traffic
Service resumed

 

 

 

A few basic dB wireless tips

If you have done a lot of wireless the below is bread and butter and appears in many text books in various guises, but for those that need a quick summary or someone looking for a quick “in” to further reading and practise, the below might clear a few things up.

First Logarithms

Why? (trust me here, this is brief and worthwhile).  Well Logarithms are useful to represent the ratio between two values (i.e. values that can be measured), and we use ratios all the time in Wireless. This is because it is so so much easier than using real numbers with many decimal places when looking at signal strength, gain and power levels.  So what are they?

Logarithms are actually fairly easy at a basic level.

For example –   how many 2s do I need to multiply to get 8?

2 x 2 x 2 = 8

So I had to multiply three 2s to get 8, so the Log is 3 – and the way you write this is below:

log1

The base is the little 2, so log base 2 of 8 = 3.  We are essentially using three numbers here:  The number we are multiplying (2 in this case), how many times it is multiplied (3 in this case), and the number we want to end up with (8 in this example).

Ok, one more just so we are clear.

Work out the following:      log2

so 6 x 6 x 6 x 6 = 1296, so we need four of the 6s multiplied to get 1296 so the answer is:

log3

Incidentally this also tells you the exponent – so 6 to the power 4 = 1296.

All well and good and a refreshing trip down memory lane but what about wireless?

Well log base 10 is used a lot in wireless, particularly when it comes to dB.

What is dB (decibel)? Well it is a ratio, and what is good at representing ratios? Well logarithms of course.  The decibel is actually a unit of measure that came out of Bell Telephone Labs, and was very useful around the attenuation of audio frequency signal down a mile long telephone cable.

The decibel (dB) therefore is a good way to express the ratio of two power levels.  A couple of equations are coming up, nothing too difficult, but hold on for the result as that is where the trick comes in to impress your friends (most people have friends who are impressed by RF conversations right?).

When you express a power ratio in decibels then it is 10 times the 10-based logarithm of the ratio.  What on earth does that mean?

We are trying to find a ratio, so for 1 Bel (B)

RatioB = log10(P1 / P0)

To take this a step further 1 Bel is 10 decibels, so that is where you get your 10 x from. Therefore for dB:

RatiodB = 10 x log10(P1 / P0)

So this gives you a handy way to work out a ratio between two real power values, and in the wireless world you would typically use this when looking at Gain (how well an antenna converts input power into radio waves headed in a specified direction.)

Let’s have a look where this is useful.

GdB = 10 log10(P2 / P1)

P2 is the power level.

P1 is the referenced power level.

GdB is the power ratio or gain in dB.

So for the gain in dB for a system with input power of 10W and output power of 20W then

GdB = 10 log10(Poutput/Pinput) = 10 log10(20W/10W) = 3.01dB

Now remember that figure 3.01dB

The first of 2 decibel values an RF engineer has etched into their head is 3dB, because as you have just seen, this is a ratio of 2 (yes we know it is actually 3.01dB, but that is close enough for RF design) – 20/10 is 2 – voila.

So this is great to work out power ratios in your head.  If you know the power level has doubled then you have a 3dB gain, if the signal level is four times higher at the output than the input you have a 6dB gain.  Equally if you have -3db then the ratio is 1/2 or a half.

The second figure to have hardwired into your brain is 10dB.  Remember dB is a ratio, and 10dB is handily a ratio of 10 🙂  Equally -10dB is 1/10 or 1 tenth.

For example, if the signal level at the output is 10 times higher than the input then you have a 10 times ratio (i.e. 10W input and 100W at output, a 10x gain) which is 10dB.

Even more usefully you can now combine the two.  Say you have an amplifier with a gain ratio of 20 ( 20 times or 10 x 2), then the gain value is 10 + 3 which is 13dB.  (3dB is a 2 x ratio).

Got it?

So now say I want to calculate the power ratio of a given value….

P2 is equal to the reference power P1 times 10 raised by the gain in GdB divided by 10.

P2 = P1  10(GdB / 10)

P2 is the power level.

P1 is the referenced power level.

GdB is the power ratio or gain in dB.

log table

Basically a positive gain means there is more power at the output than the input and a negative gain means less power at the output than the input. Now consider a 40dB gain, well that is 10000 times more power at the output than the input, whereas a 20 dB negative gain is 100 times less power at the output than the input.  You can see now where these factors of 10 and logarithms can be useful for quick calculations.

dBm

So we have established that dB is a ratio, but what about dBm?  Well this is also a ratio but a ratio to a real value, i.e. the ratio of a power level relative to 1mW (or 1 thousandth of a Watt – 0.001W).  dBm therefore is a way to express absolute power.  10 dBm then is 10mW, or 10 x 1 mW.   20 dBm is 100 mW.  Remember the factor of 10 earlier?  So 20dBm references 1mW x 10 x 10 = 100mW.

Finally you need to be careful when expressing the difference between 2 power levels. 20 dBm – 10dBm = 3dB and not dBm because again, here we are expressing a ratio between 2 values as dictated by decibels.

So why use ratios and logarithms at all?  Well look at the table below of typical 802.11 power levels and signal strength.  At -90dBm you are talking 0.000000001 mw. Calculations based on a ratio or logarithm become much easier to compare than saying “the signal strength is 0.0000something as opposed to 0.00000something so we know the power has decreased by?  and the negative gain is?”.  In essence it provides meaning to the low power levels involved in wireless communication and makes working out real-world designs a whole lot easier.

dbm to mw

So there you have it a few handy basic wireless tips.

A couple more tips to keep up your sleeve is knowing that 5db is a ratio of 3 and -5dB is 1/3 or a third.  Not exactly of course, but close enough to work things out.

dBi

dBi is another measure around antennas that is handy to keep in mind, as this is the gain with reference to an antenna that transmits evenly around a 360 degree sphere. This is the perfect antenna which radiates power in all directions equally.  Of course real antennas like this do not exist, but it can be a useful reference point when looking at antenna gain in relation to the theoretical perfect antenna.

Isotropic
Isotropic radiation pattern in free space

One of the most common antennas that you will come across is the dipole antenna, or you may have heard it called a rubber duck antenna.  This has a doughnut shaped radiation pattern (toroidal).  It is handy to know that this has a gain of 2.15dB over an isotropic antenna, so if you had a different type of antenna with a gain of 5dB, you now know this is 2.85dB of gain higher than a common dipole – 2.15dB + 2.85dB (sounds a bit like we are bird-watching now).

Dipole
Dipole antenna radiation pattern

And what is 5dB again?  Well it is a ratio, and coincidentally ratio of around 3 times the power gain over an isotropic antenna. See, I told you knowing the 5dB ratio was handy.

So remember, 3dB, 10dB, (maybe 5dB) and 2.15dBi and you can work things out like a wireless design bod in no time at all:-)

Hopefully these very basic wireless tips will send you on your way into the magical world of RF with a little less confusion when dB is thrown around like confetti.

EBITDA, Cashflow and Service Providers

It might seem odd to see a piece about cash flow in a technical blog, but looking at EBITDA and cash flow got me thinking about whether the new wave of service provision (NFV, SDN, SDO, SD-WAN etc.) has any impact on the traditional ways of reporting for Service Providers currently based on high up-front costs for future service and revenue.

For many years EBITDA has been a preferred reporting indicator of Telecommunications Providers, and in turn, Service Providers.  Execs knew the score – grow revenue, sort out your EBITDA, and get good Enterprise Value Multiples.  Enterprise Value is how much it would cost to buy a company’s business free of its debts and liabilities.  Enterprise Value Multiple is your Enterprise Value (EV) divided by your EBITDA earnings number.

But with automatic rapid growth for telecoms providers on hold for now, and the fact that even though companies may be hitting market expectations, they are not always seeing the Enterprise Value they would like in the market, the EV multiples are simply not there.

Based on that, I thought it might be interesting to look at what EBITDA is?   Why the preference exists? And do changing cycles of investment with SDO/SDN/NFV/Agile change anything at all?

——————

First up – What is EBITDA?

ebitda-cartoon

EBITDAEarnings Before Interest, Taxes, Depreciation and Amortisation.

The way to calculate this is:

EBITDA = Income before taxes + Interest Expense + Depreciation + Amortisation

So basically you add the deductions (Interest Expense, Taxes, Depreciation and Amortisation) to your net income.

Let’s break this down a little.  Before the 1980s EBIT was a key indicator of a company’s ability to service its debt (Earnings before Interest and Tax).  With the advent of Leveraged Buyouts (LBO) EBITDA was now used as a tool to measure a company’s cash. LBO is essentially a way of acquiring another company with a large amount of borrowed money.  The assets from the acquiring company as well as the assets of the company being acquired are used as collateral for the loans.  This enables acquisitions without committing lots of capital..

Some conflate operating income (or operating cash flow) with EBIT but these are not the same, as EBIT makes adjustments for items not accounted for in operating income and thus are non-GAAP (Generally Accepted Accounting Principles).  Briefly, operating income is gross income minus operating expenses, depreciation and amortization but does not include taxes and interest expenses as in EBIT.

————–

Let’s build this up from Cash Flow and Net Income.

Net income and Operating Cash Flow may seem very similar but they are not really the same.  Operating cash flow, or raw cash flow in general might need converting through accrual accounting to get a more realistic measure of income, namely net income. Accrual accounting?  Very briefly this recognises when an expense is incurred not when that expense is paid (think of buying something one month but then having 60 days before you need to pay it off, well the expense was incurred the moment you bought it really).  As a result businesses can list credit and cash sales in the same reporting period that sales were made.

Cash flow or Operating Cash Flow (OCF) = Revenue minus Operating Expenses

Ideally you want your Operating Cash Flow to be higher than your operating expenses or you might not be in business for too long.  Certainly an important measure.

Net income = Gross Revenue minus Expense.

Net income reflects the profit that is left after accounting for ALL expenditures, every pound/dollar the company earns minus every pound/dollar the company spends.

As I mentioned Cash Flow and Net Income are not the same, with Cash Flow showing not only how much a company earned and how much it spent, but when the cash actually changed hands.  This difference is significant and an illustrative example is below:

Say you sign $20million worth of contracts in a year, complete work on $10million of them, and collect $8million of that work in cash in the year.  But say you have also paid out $6million dollars in equipment, your raw cash flow would be $2million.  Net income however might look significantly different.

So if you have provided economic value to your customers (revenue) of say $15m of the contracts (i.e. you have completed $10m of the contracts, are 50% of the way through the remaining $10m, so $5m of economic value to customers), and of the $6million of equipment purchased you consume only a third of this, and they are estimated to last 3 years, then $6million divided over 3 years is $2million, so your net income for the year would be $15million minus $2 million in expenses, so $13million.

$13 million is a very different number than the raw cash flow value of $2million, and perhaps a better indication of the operating income of the company.

Earnings and cash therefore are not the same thing but that is not to say that operating cash flow is not important.  Companies must still pay interest, and they must pay taxes, and that cash has to come from somewhere, so using this to look at Free Cash Flow (Cash Flow after all capital expenditure), and looking at working capital to see whether a company can service its short term debts is useful..  A negative working capital might suggest a company will struggle with its short term debts.

Working Capital = Current Assets – Current Liabilities

Current Assets  – assets that can be converted to cash in less than a year.

Current liabilities  – liabilities that must be paid this year.

——————-

Now we have had a quick look at why Net Income might be a preferable way to get a read of a company’s operating income over and above raw cash flow, is there a way to calculate this to give us a better standard view of Net Income without all the financing decisions and conditions that are peculiar to each company? i.e. provide a better method of comparison?

Well EBITDA adds back (or deducts from the calculation the expenses associated with) interest, taxes, depreciation and amortization.  This therefore removes the effects of financing and accounting decisions.

The idea being that by ignoring expenses like interest, taxes, depreciation and amortization you strip away the costs that aren’t directly related to the core operations of a company.  The proposition is that what you are left with (EBITDA), is a purer measure of a company’s ability to make money.

Let’s take a brief look at what these costs are to reach EBITDA:

Interest expense – is the interest payable on any borrowings  such as bonds, loans, convertible debt or lines of credit (interest rate X outstanding principle amount of debt). You hear this referred to as just the Principle sometimes.

Taxes – Generally refers to income i.e. by a state or country.  Business taxes (property, payroll taxes etc. are considered operating expenses and therefore no factored into EBITDA.

Depreciation – In this sense the reduction of the value of an asset over time.  Tangible assets  – both fixed assets (land, buildings, machinery) and current assets (Inventory). Basically things that can be physically harmed.  Depreciation is a way of giving a cost to a tangible asset over its useful life, or how much of an assets value has been used up over time.

You also have something called intangible assets, which are things you can’t really touch like copyrights, patents , brand recognition etc. (as opposed to equipment, machinery, stocks, bonds and cash for example).  Goodwill is also a part of this, so solid customer base, reputation, employee relations, patents and proprietary technology – an amount you are willing to pay over the book value of the company (If the company was liquidated, the value of the assets the shareholders would theoretically receive)

Amortisation – Deals with these intangible assets.  So the paying off of debts such as a loan or mortgage with a fixed regular repayment schedule over a time period.  Also the spreading out of capital expenses of intangible assets over the assets’ useful life, i.e. a fixed period.  Say you spend $10 million on a patent with a useful life of 10 years, then over these 10 years you can spread the cost as a $1million a year amortisation expense.

———-

So EBITDA is not really a good measure of cash flow, but can be a reasonable measure of a company’s profitability or net income.  However it does leave out the cash required to fund working capital and the replacement of old equipment, which can be significant.

This is particularly useful for new companies or those trying to attack a new large market where depreciation and amortisation spreads out the expenses of large capital investments, which can be considerable.

Conversely EBITDA has a bit of a bad rep as it has been used by a bunch of dangerously leveraged companies in boom times, but that is not to say it is all bad by any means.

There are many pros and cons to EBITDA but most are out of scope of what I am trying to convey, namely that telecoms providers tend to like it as a reporting method, have large initial infrastructure outlay to get return on investment over time through telecoms or service provision.  They are also not seeing the Enterprise Value multiples they would like despite a focus on market expectations around EBITDA (which doesn’t penalise a company in the market for having to invest in longer term infrastructure providing earnings are going in the right direction).

————-

As with all accounting it is useful to look under the covers when you get chance to get a better read.  In Service Providers or Telecommunications companies where there is significant expense in building out the network, buying frequency, and installing physical towers or radios, it makes a deal of sense to judge them more fairly over a longer term as repeated income will come some time after the initial investment.

In the boom years however, rapid growth was a sure fire way of getting great EV multiples, but with growth in the industry now at GDP levels and below, fixed lines in decline, mobile revenues tailing off and there being no business case in the transport of bits, EBITDA focus does not automatically equate to investor or customer value.

In essence it gets harder and harder to hide behind growth.  Knee jerk reactions from execs can zone in on cash and EBITDA while they under-invest in the very things that provide economic value to customers (your revenue as a provider) i.e. better products and services to facilitate ROI, and lead to better capital returns.  Essentially the focus needs to be on creating value and not just controlling costs.

—————————

So my question is, with SDN, NFV, Software Defined Orchestration, Agile product development etc. can you facilitate some of the above?  Are the providers who are simultaneously adopting flexible automation and orchestration to roll out new services faster (services chained to customers’ business logic), with greater visibility and with better quality, likely to be the ones who get the EV multiples they hope for?

dinosaur

“What is that huge fiery technology shift I see?  Should we prepare?” .   “Don’t worry it’s miles away….stop thinking, keep cutting costs and we’ll be fine!”

As customers start to play providers off against each other with shorter term contracts to drive down cost and improve service (as with SD-WAN), does flexibility to respond with new services and efficient operations become more important than high sunk costs, long contracts, eventual gain, and a relentless focus on internal cost cutting with traditional technology to justify business as usual?

Is this all doubly worrying with the reduction of some SPs to mere local cloud access providers (just give me a pipe to the nearest cloud provider thanks)?  Is it coincidence that the big cloud providers are the ones physically laying fat cables to connect their services nowadays?

Reducing operating costs and quick iterative roll-outs with new techniques and technology certainly seems to appeal to the largest and more innovative SPs.  Using the latest methods to hold vendors to account and change traditional network operating practices (e.g  vCPE), while simultaneously recognizing they can provide new services to customers with more flexibility, seems primed for technology shifts like 5G.  Will all of this lead to a renewed focus on ROI rather than EBITDA growth or Capex/Sales? At the moment EBITDA growth and Capex/Sales don’t appear to be moving the EV multiple needle.

Are those who are not adopting new ways destined for extinction?

Of course EBITDA isn’t going to be ditched by Telecoms providers anytime soon, but maybe a softening of the relentless internal focus on one reporting measure will only lead to better value for customers, and in return better results for those providers focusing on customer value.  You never know, internal business cases might actually start to incorporate vision.

Architecture Reflections

 

Arch

If you follow IT news and commentary around Architectures at the moment it might seem that Enterprise Architecture is in a bit of a quandary. On the one hand there seems broad agreement that to drive IT advantages in line with business needs, at speed, a sound architectural base is needed. What is also needed is efficient processes to drive this architecture.

On the other hand there are articles talking about Enterprise Architecture being broken, or of business irrelevance of architectural frameworks as we know them today; a view of self-referential documentation which only the architecture community ever really read, while others “haven’t got time for this, there is a business to run and improve”.

As reliance on IT increases (it certainly isn’t going away), and while new models of IT consumption and service delivery are morphed all the time, I believe architecture will become more important, not less. The problem, as with so many things, is one of communication. The challenge is to communicate relevance to business stakeholders and show how we are improving or introducing new and better business relevant services, and solving business problems through architecture. There is a need to create and communicate the base architecture to achieve this.

Although the aim of this post is not to compare and contrast the different frameworks and methodologies, a brief mention of the options does highlight some of the problems we face.

We have a LOT of choice around governance, framework, service and methodologies. (COBIT, Zachmann, Togaf, ITIL, Prince2, PMBOK, CMMI, MSP, FEA, DOGAF, ISO 20000, ISO 27000, Agile, Six Sigma/DMAIC, and Lean to name but a few). Of course you will notice that, while not all mentioned are directly comparable (they overlap, and also address slightly different concerns), it does illustrate that the landscape is far from simple.

As an aside, while it is not quite accurate and too simplistic to view COBIT as the “why”, ITIL as the “how”, or with other frameworks the “how” and “what”, it can serve as a useful starting point in questioning what you can get out of each, and mapping the overlapping functions in getting them to work together.

To further illustrate the complexity let’s briefly go back to 1987 and the Zachman framework with its 4 domains and numerous sub-domains as an initial point of reference.

1) Process Domain – Business Context Engines, Planning Engines, Visualisation Engine, Business tools

2) Information/Knowledge Domain – Business Data, Business Profiles, Business Models, Data Models

3) Infrastructure Domain – Computers, Operating Systems, Display Devices, Networks

4) Organisation Domain – People, Roles, Organisational Structures, Alliances

Domains have been added in other frameworks and, as you can see, this isn’t getting any simpler even if the constructs are useful. Even a single domain can have mind boggling degrees of complexity.

If I take a component of the Infrastructure domain with which I am familiar (Network) there is a vast array of technology to architect around, each with their functional and deep technical specialists. From Software Defined Networking, Control Plane emulation and Policy creation, to layer 3 identity separation (LISP), OTV for layer 2 over Layer 3 tunneling, FabricPath and TRILL for layer 2 routing (MAC in MAC, MAC in UDP – no more spanning tree), and VXLAN (MAC in UDP) for extension of layer 2 segments over shared physical infrastructure, to name just a few recent headlining technologies. And this is just one small part of one sub-domain

You will, of course, have spotted an error in the complexity I have just outlined. A good base architecture will not have to architect around each new technology, but identify solutions to fit seamlessly into the architecture as they solve a business problem, enable a service, or support business functions. This is why architecture is there in the first place.

So we ask ourselves what business problem are we solving, what service are we enabling, or function we are supporting? For example with SDN are you gaining greater flexibility? more open platform support? better visibility? better policy control? more customisation? lower costs? better security? reduced risk? and does this let you roll out new services more quickly and robustly to serve the business? Or with some of the other technologies, are you able to move workloads faster regardless of location with less operational overhead and cost, or spin up services more quickly, reliably, cheaper? How does it aid mobility? Public / private cloud? Security? Once you ask, and indeed have your own answers to such questions, the technology seems to slot naturally into a base architecture.

Given the complexities, how do we get everyone on the same page?

We could just throw around nebulous buzz phrases like “business outcomes” and hope everyone nods in agreement, but a more practical method might yield better results.

What I am not saying is that some of this is not covered in Architecture Vision or Business Architecture phases of TOGAF for example, but it is all too easy to slip into the technicalities of the process, or explain the entire process in order to try get everyone on board with this piece, let alone contribute meaningfully.  This can often be a challenge.

One practical suggestion is briefly outlined below.

As all of the above frameworks, methodologies, processes etc. were (to a greater or lesser degree) born out of the Deming cycle (Plan, Do, Check, Act), it does allow common ground to be established and serve as a foundation of getting all stakeholders on the same page. We can use this to simplify communication and create a common understanding.

The aim is the get business stakeholders involved as early in the process as possible, to understand, and to avoid the redundancy and time wasted from erroneous requirements.

If you allow the value stream to “pull” the process and architect with this in mind, it can really help in making architectures business relevant. By this I mean viewing the process from the perspective of customer value, business value, and demand, then working backwards to eliminate waste and improve service quality. As obvious as this sounds, it is rarely done effectively.

This brings us to the option of a process/tool that can literally get everyone on the same page: the Lean A3 process/tool.

With the A3 report everything is discussed and agreed upon in one A3 sized document, which is highly visible, has agreed reference data, and follows a simple common sense process. As this process revolves loosely around the initial Deming cycle it has instant familiarity with architects, developers, designers, manufacturers and business process professionals across the board. The idea here is to get everyone to agree on the problem being solved, the service offered, or the function supported. This in turn enables a more seamless flow into the base architecture.

Although the above might indeed sound like “common sense”, and increasingly there is a reliance on this quality in architects; by formalising, and standardising this common sense in one place, with common agreed data in a concise format, with stakeholder contribution and understanding, we can then provide a solid base for the detailed architecture to really achieve what it sets out to do. It also makes it easier for anyone new, to initially understand the architecture and contribute meaningfully without wading through reams of framework documentation for weeks on end. As they say ” put talented people into average processes and you often get poor results, but put average people into great processes and you often get excellent results”.

Like anything, the A3 process/tool takes practice, (it is not something you write individually and present), but the idea of having a one-page reference that everyone has contributed to and agreed upon, can be a very powerful way of getting different functions to work together and most importantly understand why things are happening the way they are. Does it have to be the A3 process/tool? Of course not, but it does seem to be a useful reference or starting point.

Another advantage is that the components of the A3 process can quite easily be mapped to individual architecture phases in other frameworks such as TOGAF.

IT organisations will be increasingly measured by their alignment with the business; by speed and flexibility, productivity and growth, with security and risk mitigation embedded at every level. Combined with this, the idea of service managers running an IT service as a function of the business and measured as such, will be a powerful one. For me, this only puts greater emphasis on making sure everyone is referring to the same thing to avoid costly misunderstanding.

Through process/tools such as A3, allowing architectures to be pulled from the value stream, making things as simple and visible as possible, and having stakeholders embedded in the process as early as possible, maybe we can cut through some of the communication issues commonly associated with architecture relevance of late.

Some examples of the A3 process/tools can be found below:

Explanation of an A3 example – pdf

A3 example templates can be found here

Think before you leap

What Do You Know About Why You Are Doing This A3?

Is the A3 a tool, process or both?

—–

There are several formal definitions of terminology within the various frameworks. I try, where possible, to ground myself in standard English definitions of the terminology firstly to remind myself of what I am trying to achieve, and secondly to gain common understanding. Some of these basic dictionary definitions are included below:

TOGAF defines architecture as

  1. 1. A formal description of a system, or a detailed plan of the system at a component level to guide its implementation
  2. The structure of components, their inter-relationships, and the principles and guidelines governing their design and evolution over time”

Dictionary definitions

Architecture – the complex or carefully designed structure of something

Architect – ” a person responsible for inventing or realizing a particular idea or project.” from the Latin or Greek arkhitekton meaning “chief – builder”

Framework – a skeletal structure designed to support or enclose something – a frame or structure composed of parts fitted together, or a set of assumptions, concepts, values, and practices that constitutes a way of viewing reality