A few years ago I wrote about the new industry phenomenon where the big users of routers and switches like Facebook, Google, Microsoft, and Amazon were saving huge amounts of capital by buying generic routers and switches and writing their own operating software. Since those early days these companies have also worked to make these devices far more energy efficient. At the time of that blog, I noted that it was impractical for smaller ISPs to take advantage of the cheaper gear because of the difficulty and risk of writing their own operating software.
That’s all changed and there now is a viable way for smaller ISPs to realize the same huge savings on routers and switches. As you would expect, vendors stepped into the process to match whitebox hardware and operating software to create carrier-class routers and switches for a fraction of the cost of buying name brand gear.
There are a few new industry terms associated with this new industry. Whitebox refers to network hardware that uses commodity silicon and disaggregated software. Britebox refers to similar hardware, but which is built by mainstream hardware vendors like Dell. Commodity hardware refers to whitebox hardware matched with mainstream software.
There are a number of vendors of whitebox hardware including Edge-core Networks, SuperMICR, Facebook, and FS.Com. Much of the gear is built to match specifications provided by the big data center operators, meaning that hardware from different vendors is now becoming interchangeable.
The potential savings are eye-opening. One way to look at the cost of switches is to compare the cost per 10-gigabit MPLS port. In looking at list prices there is a whitebox switch available from FS.Com for $92 per port; Dell britebox hardware is $234 per port; Juniper is priced at $755 per port and Cisco at $2,412 per port. To be fair, a lot of buyers get discounts from the list prices of name brand hardware – but a 96% savings over list price is something that everybody needs to investigate.
As I mentioned, the whitebox hardware is also more energy efficient – saving money on air conditioning is the issue that led the big data center companies looking for a better solution. The FS.Com 10 gigabit switch uses about 200 watts of power annually; the Dell Britebox uses 234 watts; the Juniper switch uses 650 watts and the Cisco switch uses 300 watts. There is no question that a whitebox solution is greener and less expensive to operate.
There is also off-the-shelf software that has been created to operate the whitebox hardware. The most commonly used are IP Infusion OcNOS and Cumulus Linux. The cost of this disaggregated software is also far less expensive than the cost of the software embedded in the cost of mainstream hardware.
Probably the biggest concern of any ISP who is considering a whitebox solution is configuring their system and getting technical assistance when they have problems. The good news is that there are now vendors who have assembled a team to provide this kind of end-to-end support. One such vendor is IPArchiTechs (IPA). They have engineers that will configure and install a new system and a 24/7 helpdesk that provides the same kind of support available with name brand gear.
There are other advantages for using whitebox hardware. Should an ISP ever want to upgrade or change software the hardware can be reused and reprogrammed. The same thing goes with the disaggregated software – an ISP licenses the software and can transfer it to a different box without having to ‘buy’ the software again. The whitebox software also avoids upgrade fees often charged by vendors to increase speeds or to unlock unused ports.
There is whitebox gear available for most ISP functions. In some cases the same gear could be used in the core, in aggregation points or in the last mile just by changing the software – but there is whitebox hardware sized for the various uses. There are still a few network functions that the whitebox software hasn’t mastered, like BGP edge routing – but the hardware/software combination can handle the needs of most ISPs I work with.
Whitebox hardware and software has come of age. Anybody buying expensive Cisco or Juniper gear needs to consider the huge savings available from a whitebox solution. The big vendors have been successful by forcing customers to pay for numerous features and capabilities they never use – it makes more sense to buy more efficient hardware and pay for only the features you need.