Cognificance

Cognificance

About the significance of machine cognition

Cognificance

About the significance of machine cognition

Google's TPU 2.0

Google made their AI framework TensorFlow open-source in late 2015. Most AI frameworks use relatively inexpensive and widely available GPU (Graphics Processing Unit) devices to accelerate AI-related filtering, as doing this with regular CPUs isn't very cost effective (Watts/Output). But even GPU's - while much more efficient that CPU's for this type of activity - can be beat by differentiated hardware.

Research into making AI-related computing more efficient has shown that "bittedness" isn't what drives performance. While modern operating systems require 64-bit processors for efficient work, deep learning machines are quite happy to work with 8 bits, as long as there are lots and lots of nodes to avoid swapping.

Google research developed a
Tensor Processing Unit (TPU) to work alongside their TensorFlow framework. The device was designed to plug into a regular hard-drive slot, making roll-out of large clusters of TPUs quite simple: set up server racks with slot-in hard drive capacity and plug these (mostly) full of TPUs.

The advantage of the TPU over a GPU-based solution is a much higher number of operations per second per watt. I.e.: faster number crunching with lower power requirements. Google's TPU was never made available to the market, though I wouldn't be surprised to find something similar - albeit much scaled down - in a Google Android phone in a few years.

Things don't stay at a standstill in IT, especially not at Google, so it comes as no surprise that the
successor of the TPU has been announced. This "TPU 2.0" (also dubbed "Cloud TPU") device doesn't offer the same form factor as version 1. Just looking at the towering heat sink gives you a feeling that there is quite a bit of neural capability waiting to be unleashed.

And indeed: while the original TPU could only be used to run pre-trained neural networks, this new version is designed to facilitate an efficient learning cycle as well. According to Google, the TPU 2.0 can train a neural net several times faster than comparable GPU farms.

The TPU 2.0 was designed specifically for a Cloud-based offering. In other words: anyone can put together an AI solution using Google's open-source TensorFlow framework and run this solution on the Google Cloud with access to TPU 2.0 farms. All at a price, of course. Will this be a success for Google? In my opinion, selling TPU time via Cloud-based AIaaS (AI as a Service) isn't the prime objective of all the R&D that has gone into this new device. Google itself has transformed to an AI company, with most services it offers, from Maps to Search to Pictures all using AI in some form. Not to forget Google Home - the associated service requires intense AI processing for natural language processing (NLP) of voice input.

As the world moves to AI - and who wouldn't like to have an intelligence built into their "Smart"-Phone - you can bet your booties that companies like AMD, Intel or Nvidia are hard at work, designing industrial or even consumer-grade AI hardware. The next two years will likely show a plethora of TPU-like processing devices coming to a computer store near you!
Comments