According to Kostadis, File Virtualisation is Snake Oil. I'm sure I can find at least a couple of vendors who would disagree with him!
I think it is the scale thing; Kostadis talks about tracking millions of files and moving them about; external meta-databases to keep track of the objects I mean files. Vendors like NetApp are going to have to grasp this nettle of scale and work with it.
For Cloud to work and to gain the scale it requires long term; this is going to have to happen. A file is just a specific type of object and Cloud File Stores have been designed in general to deal with millions of objects. Not all of the problems have been solved but these are not impossible problems to solve and it is not even improbable that they will be solved.
Also, I take issue with his comments about deduplication; file virtualisation can enable deduplication, not at the block level but at the file level. Your gains might not be as good but you can single instance files; these files might have been compressed using technology such as that provided by Ocarina.
The things you do potentially loose are those things which are storage-array based functionality but by externalising some of that capability and becoming truly virtual; your gains may outway that which is lost.
BTW, I still think DataMotion is cool; anything which allows data to move between storage-systems without outages is cool. The easier we can make this, the better! But there are many ways to skin a cat and there will be many useful solutions to the problems that we face.
Reminds me of the entry at Chuck’s Blog about the future of filesystems. (http://chucksblog.emc.com/chucks_blog/2009/08/the-future-doesnt-have-a-file-system.html)
There’s going to have to be some change in the way things are organized. Logical subdivision will only work for so long.
There will be different file-systems/stores for different purposes. There will be no one true file-system but there never has been although arguably for a whilst we have been bumbling along with good enough file-systems but I think scale means that good enough won’t be good for much longer for many things.
I find it interesting when people confuse File Virtualization with object stores.
File Virtualization is the name given to a suite of products that are bumps in the wire that are layered on top of file systems. Products that come to mine are Rainfinity or Acopia.
Those products don’t work, don’t scale, and don’t perform.
Object store kind of infrastructures, are just another way to build a file system.
The traditional file system architecture embedded the namespace and the object store into a single piece of code. To do a lookup of a file, you first had to find the inode that was stored in a directory structure on disk.
However, it’s quite clear that that was a convient simplification.
The kinds of file systems you’re talking about (and they are file systems) just build the directory lookup outside of the storage system.
I must be a dinosaur.
kostadis
When you say don’t work? What do you mean? Because I suspect that they do work; they may not deliver all the benefits that they promise and deployment may not be as simple as portrayed but don’t work? That’s a little strong.
When vendors point at a competing product and say that they don’t work; I actually tend to switch off! I know plenty of us customers who do the same thing! You see, don’t work==FUD!
And of course, then there is the problem with the ‘V’ word; we (including customers) can make it mean what-ever we want it to! 😉
And as for dinosaur? If you really want to be but I don’t put you in that class at all!
martin,
File Virtualization works in limited environments, but the promises made by the vendors so eclipse their technology capabilities that I feel it’s okay to say they don’t work.
But good point about “don’t work == FUD”. I think I’ll keep that in mind when I write in future.
cheers,
kostadis
@kostadis
For a pretty smart guy, you’re saying some pretty dumb things.
As you know, EMC sells file virtualization in the form of Rainfinity. Trust me, it works as advertised, and has done so in thousands of accounts around the globe. To argue otherwise would put you in the Flat Earth Club.
Can it be over-hyped or over-sold? Of course. Same thing can happen to NetApp products as well, and does all the time.
Does that mean that NetApp products “don’t work”?
Of course not.
— Chuck