Infrastructure Fan-Out

In a previous post I discussed EMC CTO John Roese's use of the phrase application fan-out. His definition describes a common industry trend: a vast array of applications can be placed in an increasingly large set of application frameworks, which can then be placed in any number of virtualization frameworks. This "fan-out" is depicted below.

To make matters more complex for data center architects, the underlying layers of the infrastructure have fanned-out in a similarly complex manner:

Let's step through these layers from the bottom up:

• Protocol: Applications are increasingly leveraging protocols that go beyond block and file, whether it be an HDFS approach or an object-based method (note: up next will be memory-based access to storage).
• Media: The application requirements for storage media are increasingly varied based on how frequently (e.g. hot edge SSD) or infrequently (e.g. cold storage) the data is accessed.
• Infrastructure: While there is clearly a trend, for some application workloads, to move from appliance-based HW to commodity storage, there is also a growing use case for custom non-volatile storage that appears as an extension of the memory bus.
• CPU: While x86 is the predominant CPU architecture, ARM and GPU are also appearing in data centers with regularity.
• Topology: Many data center operators have achieved an effective private cloud topology; more and more often these data centers must integrate with public cloud and/or TELCO topologies as well.

Consider a new application service that might get deployed into this type of environment:

A company wants to add a new feature to their mobile phone application to accept photos and/or videos from customers, along with accompanying "notes" . The data itself is important but not mission critical. There is little reason to govern the data. It is possible that the submission of videos/photos could spike at any time. One valid option is to create a new back-end service that accepts these downloads, package the service in a VM, and then deploy the VM in a PaaS framework like CloudFoundry.

This app would then balance against an HDD-based object store (capable of storing the content and the metadata) running on a commodity appliance. It can be actually hosted by an external service provider that has implemented on orchestration framework such as OpenStack.

If I were to draw a line highlighting each specific selection of capabilities across these nine different layers, it would look something like this:

The dynamic construction of this type of stack is essential as the number of users, applications, and data continues to scale unabated.  There are M layers with a variety of choices (a0-aN) in each layer. The permutations are endless, and manual stack construction by a human being is too slow, error-prone, and impossible to modify in the face of change orders and/or configuration changes.

In a previous post I hinted that a mathematical approach for stack orchestration is really the best solution.  

In an upcoming post I will revisit this topic in the context of application and infrastructure fan-out.

Steve

http://stevetodd.typepad.com

Twitter: @SteveTodd

EMC Fellow