Yale researcher warns of cloud complexity

A Yale researcher is warning that cloud based systems could melt down as the systems become more complex.

According to Network World, Bryan Ford wrote in a paper, which he will present to the USENIX HotCloud 2012 conference this week, that as the use of cloud computing becomes more and more mainstream, serious operational “meltdowns” could arise.

The problem is that as end-users and vendors mix, match and bundle services everything will get very complex and where there is complexity there is an accident waiting to happen.

He said that as diverse, independently developed cloud services share more fluidly and aggressively multiplexed hardware resource pools, there will be some strange things that will happen.

These include unpredictable interactions between load-balancing and other reactive mechanisms which could lead to dynamic instabilities or ‘meltdowns.’

Ford said it was a bit like the intertwining, complex relationships and structures that helped contribute to the global financial crisis.

He said that new cloud services may arise that essentially “resell, trade, or speculate on complex cocktails or ‘derivatives’ just like the modern financial and energy trading industries”.

Each of these components will be maintained and deployed by a single company that, for reasons of competition, shares as few details as possible about the internal operation of its services, Ford said.

What will then happen is that the cloud industry could face speculative bubbles.

There will be occasional large-scale failures because composite cloud services have weaknesses that don’t become known until the bubble bursts.

Ford has no idea how to fix the problem. But he suggests providers should release detailed information about their system dependencies to “a trusted third party offering cloud reliability analysis services”.