Storagebod Rotating Header Image

April, 2016:

Pestilential but Persistent!

There is no doubt that the role of the Storage Admin has changed; technology has moved on and the business has changed but the role still exists in one form or another.

You just have to look at the number of vendors out there jockeying for position; the existing big boys, the new kids of the block, the objectionable ones and the ones you simply want to file. There’s more choice, more decisions and more chance to make mistakes than ever before.

The day-to-day role of the Storage Admin; zoning, allocating LUNs, swearing at arcane settings, updating Excel spreadsheets and convincing people that it is all ‘Dark Magic’; that’s still there but much of it has got easier. I expect any modern storage device to be easily manageable on a day-to-day basis; I expect the GUI to be intuitive; I expect the CLI or API to be logical and I hope the nomenclature used by most players to be common. 

The Storage Admin does more day-to-day and does it quicker; the estates are growing ever larger but the number of Storage Admins is not increasing in-line. But that part of the role still exists and could be done by an converged Infrastructure team and often is. 

So why do people keep insisting the role is dead? 

I think because they focus on the day-to-day LUN monkey stuff and that can be done by anyone. 

I’m looking at things differently; I want people who understand business requirements who then turn these into technical requirements who can then talk to vendors and sort the wheat from the chaff. People who can filter bullshit; the crap that flies from all sides; the unreal marketing and unreal demands of the Business.

People who look at complex systems and can break them down quickly; who understand different types application interactions, who understand the difference between IOPS, latency and throughput.  

People who are prepared to ask pertinent and sometimes awkward questions; who look to challenge and change the status-quo. 

In any large IT infrastructure organisation; there are two teams who can generally look at their systems and make significant inferences about the health, the effectiveness and a difference to the infrastructure. They are often the two teams who are the most lambasted; one is the network team and the other the storage team. They are the two teams who are changing the fastest whilst maintaining a legacy infrastructure and keeping the lights on. 

The Server Admin role has hardly really changed…even virtualisation has little impact on the role; the Storage and Network teams are changing rapidly, many are embracing Software-Defined whilst the Industry is trying to decide what Software-Defined is.

Many are already pretty DevOps in nature; they just don’t call it that but you try to manage the rapid expansion in scale without a DevOps type approach. 

I think many in the industry seem to want to kill off the Storage specialist role; it is more needed than ever and is becoming a lot more key…you probably just won’t call them LUN Monkeys any more..they’ve evolved!

But we persist…

Reality is persistent

I see quite a few posts about this storage or that storage..how it is going to change everything or has changed everything. And yet, I see little real evidence that storage usage is really changing for many.  So why is this? 

Let’s take on some of the received wisdom that seems to be percolating around. 

Object Storage can displace block and file?

It depends; replacing block with object is somewhat hard. You can’t really get the performance out of it; you will struggle with the APIs especially to drive performance for random operations and partial updates.

Replacing file with object is somewhat easier, most unstructured data could happily be stored as object and it is. It’s an object called a file. I wonder how many applications even using S3  APIs treat Object Storage anything other than a file-store, how many use some of the extended metadata capabilities? 

In many organisations; what we want is cheaper block and file. If we can fake this by putting a gateway device in front of Object Storage; that’s what we will do. The Object vendors have woken up to this and that is what they are doing. 

But if a vendor can do ‘native’ file with some of the availability advantages of a well-written erasure coding scheme at a compelling price point, we won’t care.

And when I can boot from Object Storage..call me.   

All new developers are Object Storage aficionados?

I’m afraid from my limited sample size; I find this is rarely the case. Most seem to want to interact with file-systems or databases for their persistence layer. Now the nature of the databases that they want interact with is changing with more becoming comfortable with NoSQL databases.

Most applications just don’t produce enough data to warrant any kind of persistence layer that requires Object or even any kind of persistence layer at all.  

Developers rarely care about what their storage is; they just want it to be there and work according to their needs. 

Technology X will replace Technology Y

Only if Technology Y does not continue to develop and only if Technology X has a really good economic advantage. I do see a time when NAND could replace rotational rust for all primary storage but for secondary and tertiary storage; we might still be a way off. 

It also turns out that many people have a really over-inflated idea about how many IOPs their application need; there appears to be a real machismo about claiming that you need 1000s of IOPS…when our monitoring shows that someone could write with a quill pen and still fulfil the requirement. Latency does turn out to be important; when you do your 10 IOPS, you want it to be quick. 

Storage is either free or really cheap?

An individual gigabyte is basically free; a thousand of these is pretty cheap but a billion gigabytes is starting to get a little pricey.

A Terabyte is not a lot of storage? 

In my real life, I get to see a lot of people who request a terabyte of storage for a server because hell, even their laptop has this amount of storage. But for many servers, a terabyte is a huge amount of storage..many applications just don’t have this level of requirement for persistent data. A terabyte is still a really large database for many applications; unless the application developers haven’t bother to write a clean-up process.

Software-Defined is Cheaper? 

Buy a calculator and factor in your true costs. Work out what compromises you might have to make and then work out what that is worth to you. 

Google/Amazon do it, so we can too?

You could but is it really your core business? Don’t try to compete with the web-scale companies unless you are one..focus on providing your business with the service it requires. 

Storage Administration is dead?

It changed, you should change too but there is still a role for people who want to manage the persistent data-layer in all it’s forms. It’s no longer storage…it’s persistence.

Mine is the only reality?

I really hope not…