As the unit cost of storage continues to trend to zero and that is even with the current premium being charged due to last year’s floods; how do we truly measure the cost of storage and it’s provision?
Now, many of you are now thinking ‘zero’? Really?
And my answer would be that many of the enterprise class arrays are down to a few dollars per gigabyte over five years; certainly for SATA and NL-SAS. So the cost of storing data on spinning rust is heading towards zero; well certainly the unit cost is trending this way.
Of course, as we store ever more data; the total cost of storage continues to rise as the increase in the amount of data we store outstrips the price decline. Dedupe and the likes are a temporary fix which may mean that you can stop buying for period of time but inevitably you will need to start increasing your storage estate at some point.
So what are you going to do? I don’t think that we have a great solution at the moment and current technologies are sticking plasters with a limited life. Our storage estates are becoming like landfill; we can sift for value but ultimately it just hangs around and smells.
It is a fact of life that data management is a discipline much ignored; lots of people point at the Cloud as a solution but we are simply shifting the problem and at some point that’ll come unstuck as well. Cloud appliances will eventually become seen as a false economy; fixing the wrong problem.
Storage has simply become too cheap!
Which is a kind of odd statement for Storagebod to make….
I think it’s a useful thought exercise that you propose.
If storage (hardware) was “free”, how would you do things differently? The answer is simple: you’d focus more on value generation around information, and less about getting the last unit of efficiency out of your hardware.
And that might be a good thing.
— Chuck
In the past, data management was done by users themselves. They just couldn’t afford waiting several minutes to transfer a file via email for example, and they thought of ways to keep the files small.
With growing network throughputs, data management was shifted from the user to the IT Admin who “oversaw” total storage operations of his/her user base.
Today, exploding data growth is just normal for everybody. It is totally ok that users store twice as much data as the year before, or even more. Nobody asks why this is necessary, it is accepted as truth, but it is totally wrong.
A lot, and I mean a real lot of the data stored is NOT information. Because of the vicious trend that storage becomes cheaper and cheaper, no application vendor, no camera manufacturer and especially no storage vendor gives a penny about efficient data management.
But when you think of technology like Native Format Optimization (NFO) which reduces single data footprints by 50-90%, are you really still willing to accept the fact that data in your organization grows “explosively”?
If food were available at zero or really low cost, would you also eat as much as you can or still try to stay lean and healthy? You stay lean because you don’t want to build up hidden (expensive) health risks. You might want to do the same for your storage via NFO.
@ Chuck
That is a very interesting thought exercise. If storing data would be “free”, more value creation would indeed be done around simple storing. It would be around accessing, retrieving, assembling and syncing information. All happens with either files or database queries. What then matters is speed of accessing information, convenience and intelligence. Reducing the data footprint of file sizes then still matter a lot! And again, pushing systems to perform high I-O activities is just working on the symptoms, not the cause. So, NFO makes again a ton of sense even if storage was free…
If food was free, you wouldn’t start eating all you could eat, just because it is free, right? You would look at how you can eat better, healthier and in a more pleasant environment, etc. Same story.
Only one answer – ‘Delete’.
But of course we don’t. Just look at your own home repository of ever larger digital photos (or MP3s or AVIs or …)
Once again, we probably have to look to the computers to do it for us – “Useful Void: The Art of Forgetting in the Age of Ubiquitous Computing” http://web.hks.harvard.edu/publications/workingpapers/citation.aspx?PubId=4830
Define it, legislate it and then let them forget.
– Alastair