Data Warehouse Applications

Teradata has announced the latest in its line of data warehouse appliances (DWA). This is the Teradata Extreme Performance Appliance 4555: it’s a very specialised beast and something of a showcase.

It's so specialised as to be unsuitable for most data warehouses and is unlikely to appeal to the mass market. It will, however, have irresistible appeal and give unbeatable performance in certain specific data handling situations.

The new DWA uses all solid state disks (SSDs). In the past SSDs have often failed to live up to our expectations: they weren’t as fast as they should have been and issues with reliability and longevity kept raising their heads. It;s now generally felt that these problems have been solved. Teradata in particular has conquered the speed issue and quotes astonishing figures for the Extreme Performance Appliance of 55,000 IOPS/sec (input/output operations per second).

So why won’t we all rush to buy one? The average data warehouse contains a relatively small proportion of data that is accessed frequently (termed 'hot' data by Teradata) and a large proportion of data accessed infrequently ('cold' data). If your data warehouse falls into the 7-200 terabytes range supported by the Extreme Performance Appliance you could certainly use one and, yes, it would run very very quickly.

But the very nature of the access pattern to the average data warehouse ensures that you would be wasting most of the EPA's (sorry, it's just too long a name) phenomenal performance. You wouldn’t be using it to best advantage and it wouldn’t be a cost effective solution.

Nevertheless there are certain cases where it would be the ideal platform, cases where you have a relatively small data set (small in Teradata terms – large in terms of other BI companies) which is heavily accessed in its entirety. This is classic data warehouse application territory.

An example would be a month’s worth of network traffic data upon which you wish to perform deep analysis, rolling over the data at the end of each month. You’d get the performance without having a negative impact on the rest of the organisation.

For more mainstream use in a data warehouse a hybrid machine might prove more effective, using both SSD and hard disk storage and making use of Teradata’s existing technology for migrating hot data onto fast storage. It’s my guess that this is what the company will offer soon – watch this space.

There’s not much information about price out there yet – Teradata won’t have the kit for sale until the first quarter of 2010 – but the real question is one of cost effectiveness in the true meaning of the words (and not, in this case, as a euphemism for 'cheap').

Essentially the EPA is going to be about four times as expensive as the rotating media equivalent (ouch). But then, it is about eight times faster (yummm!) which makes it twice as effective in terms of the performance that you get for the price. So if you need what the Extreme Performance Appliance can do, it’s an economic way of getting that performance. The showcase is not just for show: it is likely to be extremely good for specialised applications.

Next page: Teradata and Data Warehouse Applications