Must be time for me to witter on about some Cloud thoughts again, someone put up a slide at CloudCamp London last week basically saying something along the lines that in the Cloud all ‘Servers are Equal’; you don’t get a choice apart from number of CPUs or amount of Memory; that’s about as granular is gets.
So all the servers are equally likely to fail, you don’t get any real options at the server-level for building in a resiliency, redundancy; even I/O performance. You can’t ask for better network connectivity and you certainly can’t ask for faster storage; sure you can have more storage but that storage will perform pretty much the same as anybody else’s and that performance may change from day to day; hour to hour; minute to minute; application developers are going to have get pretty smart about how they handle changes in responsiveness from their back-end data-stores.
But perhaps automated storage tiering will come to the rescue? And of course this lets service providers come up with yet another charge; not only a charge for space consumed but perhaps also a charge for the number of IOPs consumed over a period of time. This also changes application design, not only do you want to engineer applications to cope with variances latency but you also want to design applications to minimise I/O. Arguably applications should be designed like that anyway but in the future, there might well be real cost incentives to do so..
Just thinking..
Good thoughts Martin….now that the cloud is starting to be looked at in serious discovery mode by organisations and additionally shoved down organisations throats with the vendor marketing machine, are we starting to see where a Public Cloud differs to the good old fashioned internal datacentre (Or Private Cloud), from which in the later we have the flexibility to provide our organisational performance requirements and availability at the Infrastructure layer?
Yes im sure EC2 would be quite capable of providing various high performant storage configs, highly resilient and clustered servers and all that jazz….but at a price point which suddenly becomes similar or more when compared to that good old fashioned internal datacenter (or private cloud) 🙂
Okay, I’m going to come across as an Infrastructure Bigot now…so turn away now! But what we are seeing is not that uncommon…
1) Lots of complaints about infrastructure..lots of ideas about what infrastructure should look like…
2) Infrastructure guys of all kinds move heaven and earth to build this promised land….
3) Milk and honey fester for some time…
4) Eventually someone turns up and mutters ungratefully…and has greek yoghurt with honey for breakfast but complains that they really wanted the traditional bacon and eggs all along!
The number of times that the infrastructure is in place way before anyone is ready to use it is actually quite staggering IME. And there is more to Cloud/Dynamic DataCentres than the ability to throw up a few LAMP servers quickly.
There is going to need to be some significant changes in Enterprise level development as well. If we take Nicholas Carr’s analogy of compute power as a electricity, it’s bloody useless if all we’ve got are gas lamps!
In no way am I anti-Cloud but at the moment, I wonder when we are going to see real innovation.
“You can’t ask for better network connectivity and you certainly can’t ask for faster storage; sure you can have more storage but that storage will perform pretty much the same as anybody else’s and that performance may change from day to day; hour to hour; minute to minute;”
It won’t perform the same, it will perform poorer.
I spent a few weeks troubleshooting an IO performance issue. One of the discoveries was remote
writes were taking an additional 4 ms, regardless of
when they occured. I let the client know and speculated as to why.
The remote synchronous fibre connected storage was
at the DR provider. Fat pipeline (dark fibre?). I
could go on speculating as to why, but perhaps the
equipment from A to B was the overhead as the exact
same local storage was 4 ms faster on writes (low ms
range). DR site about 5 miles away (not much of a DR, eh?)
Storage in the cloud? Guarantee you there will be
more than 4 ms overhead on remote writes. Knee-cap
your writes, users will scream when they are used to
Average Write Times, Max Write Times that look like
this (write-back cache in play. Looks better prior to posting, Average write times are 1 ms or less):
Disk Busy% KBPS TPS AWT MWT AQW AQD
hdisk31 6.7 8.1K 125.7 0.8 0.8 0.0 0.0
hdisk27 3.3 130.0 21.8 0.8 2.1 0.0 0.0
hdisk28 2.0 866.4 54.3 1.0 1.3 0.0 0.0
hdisk26 0.7 69.6 4.5 1.0 2.1 0.0 0.0
hdisk34 0.4 7.2 1.7 0.7 1.1 0.0 0.0
The answer?
Move the whole ball of wax to The Cloud.
Servers local and storage remote and you plan on
synchronous IO remotely?
Warning: Pain ahead!