KML_FLASHEMBED_PROCESS_SCRIPT_CALLS
 

Cloudcenters are Datacenters in the Sky

January 8th, 2009 by - 32,163 views

Amazon’s Web Services (AWS) is not the only way to build scalable Cloud Infrastructures.  There are two emerging methodologies for constructing Infrastructure-as-a-Service (IaaS) AKA “Cloud Infrastructure Services”.  The first is what we call “cloudcenters”, which are essentially datacenters in the sky.  The second is what we call an “Infrastructure Web Service”.  GoGrid was one of the pioneers for cloudcenters, while AWS largely created the second model.

New_Cloud_PyramidUnderstanding IaaS means looking closely at these two approaches.  Clearly the notion of cloudcenters embodied by AWS competitors such as ourselves, FlexiScale, ElasticHosts, AppNexus, and others is important.  My colleague, Michael Sheehan, will go into more depth on how we think this distinction modifies his earlier Cloud Pyramid (right) in a follow-on blog posting to this one.

Infrastructure Cloud Models

Understanding these two approaches is important because it directly affects your selection of a Cloud Infrastructure provider.  These two models highlight a difference in core infrastructure and in target markets. Cloudcenters provide a direct equivalent to traditional datacenters and hence are usually more desirable for IT staff, systems operators, and other datacenter savvy folks.  Infrastructure Web Services on the other hand are more analogous to Service-Oriented-Architectures (SOA), require significant programming skills, and are much more comfortable for software developers.

Infrastructure Web Services

I’m going to assume for this article that you are somewhat familiar with Amazon Web Services (AWS), but I’ll briefly re-cap.  AWS provides a number of different ‘Web Services’ that can be consumed individually or put together to support different kinds of applications, usually a batch processing or web application of some kind.  These services include:

  • Object-based file storage via Simple Storage Service (S3)
  • Servers on demand via Elastic Compute Cloud (EC2)
  • Block storage on demand via Elastic Block Storage (a part of EC2)
  • Distributed database functionality via SimpleDB (SDB)
  • Content distribution and edge-caching via CloudFront
  • Messaging & queuing via Simple Queuing Service (SQS)
  • Payment processing via Flexible Payment Services (FPS)
  • Billing & re-packaging of the above services via Amazon Dev Pay

This is a robust ecosystem of services from which you can use any or all in order to build your application, getting the traditional benefits of Cloud Computing such as self-service, pay-as-you-go, and massive scalability.

Unfortunately, every service above is based on an Amazon standard, not an industry standard.  S3 is not accessible via CIFS or NFS.  EC2 provides Xen hosting, but image management and storage is completely custom.  SQS does not use any standard queuing or messaging protocols such as JMS or STOMP.  SimpleDB now has an ‘SQL-like’ interface, but is essentially a 100% ground up creation of Amazon.

A major advantage of using the Amazon approach however is that greenfield applications developed from scratch have a very powerful set of vetted, scalable services, that can be used to build that application.  This means avoiding the intrinsic and extrinsic costs associated with deploying a separate queuing or database system.

The alternative, of course, is to use the same tools, paradigms and standards that you deploy in an industry standard Enterprise datacenter today.

What’s a Cloudcenter?

Cloudcenters are that solution.  They provide the same kinds of tools that all datacenter and server operators are already accustomed to, but with all the traditional advantages of cloud (i.e. self-service, pay-as-you-go, and scalability).  Instead of creating completely new paradigms, cloudcenters are a methodology by which you, the customer, can have a virtual datacenter hosted ‘in the sky’.  Each virtual datacenter, a ‘grid’ in GoGrid terminology, is hosted in isolation from other customers, in a cloudcenter as shown in the diagram below:

cloudcenter-diagram

Your grid is akin to a traditional datacenter whereby you have all of the regular components you expect such as hardware firewalls[1], hardware load balancers, network storage (NAS or SAN[2] ), virtualized servers, dedicated networks (VLANs), and the option for physical servers for workloads that should not be virtualized.

Logically, a grid looks like:

gogrid-customer-deployment-diagram-dbconnect2

As you can see, this looks very much like a traditional datacenter infrastructure.

Traditional Datacenters and The Cloud

Just briefly I want to highlight what is in a ‘traditional’ datacenter.  These are all built in the same way.  The following diagram highlights the typical components in a datacenter and their relative dependencies on each other.  (This diagram isn’t perfect, it’s simply meant as a talking point.)

traditional-datacenter-stack

Cloud Infrastructure providers are in the business of providing you the equivalent functionality in the cloud using their scale for cost-effective service delivery.  They must also package this functionality to provide you a high level of control as it’s no longer your datacenter.  Control comes through a user interface (GUI), programming hooks (API), transparency (SAS70 audits), and performance and reliability guarantees (SLAs).

Cloudcenters focus on making your Cloud Infrastructure look very much like infrastructure you already have or are already familiar with, while Infrastructure Web Services ask you to embrace a new paradigm.

Cloudcenter Advantages

Besides the obvious advantage of ‘looking like’ your current datacenter, cloudcenters allow for strategies like using the Cloud for off-site disaster recovery.  It will be much easier to model a copy of your current datacenter to a cloudcenter than it would be to model a copy onto a Infrastructure Web Service.  There are many other advantages to the cloudcenter approach such as:

  • Servers meant to be reliable not ‘unreliable-by-design’ on commodity hardware
  • Cloudbridging[3] will be much easier between dedicated VLANs on each side using IPSEC VPNs
  • Cloudcenters will look more and more like each other over time, while there will probably never be another AWS
  • Datacenter software tools will ‘just work’ with datacenter & cloudcenter standards
  • User Interface for controlling the entire datacenter including firewalls and loadbalancers, not just servers[4]
  • Let your cloudcenter provide core infrastructure services like DNS, DHCP, NTP, and image management

Cloud Choices

Ultimately it’s great that there is so much choice when selecting a Cloud Infrastructure provider.  Both Amazon and GoGrid provide compelling solutions.  We believe that GoGrid, the very first public cloudcenter, is the Cloud of choice for sysadmins, operators, and Enterprise IT staff who need infrastructure that looks just their current datacenter infrastructure.

In the future we’ll blog in much more detail contrasting and comparing these two approaches to building Cloud Infrastructure services.

  1. To be released in 1Q09 []
  2. Scheduled for 2009 []
  3. The ability to transparently connect you internal datacenter to your external virtual datacenter or cloudcenter []
  4. Imagine VMware VirtualCenter++ []

14 Responses to “Cloudcenters are Datacenters in the Sky”

  1. [...] your eyes on GoGrid and read the Cloudcenters post right now to understand how a Cloudcenter looks remarkably similar to your existing data [...]

  2. Randy Bias says:

    Hi Scott,

    Thanks for the question and the opportunity to clarify. Yes, like a datacenter, a cloudcenter focuses on reliable servers, not unreliable servers. In a typical datacenter, you might have VMware deployed and it's expected that your VMs can be started, stopped, moved, etc. without any ill effects. AWS currently defines what an Infrastructure Web Service looks like and their servers are assumed to be unreliable. Until such time as we see a different model it seems safe to assume that's the de facto model for non-cloudcenters.

    Best,

    –Randy

  3. Thanks for the nice post, Randy. I'm not clear on what you mean by "Servers meant to be reliable not ‘unreliable-by-design’ on commodity hardware." Does the Cloudcenter approach somehow imply more redundancy or higher MTBF by design?

  4. Vijay says:

    Interesting article. Are there any specific system management challenges which you anticipate in a Cloudcenter which are different from a Datacenter.

    • Randy Bias says:

      Vijay,

      Thanks for asking. Cloudcenters are obviously *not* datacenters so there will be quite a few areas of divergence. Some that come to mind off the top of my head:

      – Less control over border network infrastructure (i.e. you don't control the upstream routers, only your switch and load balancers)
      – You will have to replace network automated installation systems like Kickstart/Jumpstart with virtual machine image mgmt processes
      – Need for robust command line scripting tools to allow sysadmins to deliver the automation they are accustomed to
      – Access to the 'console' is usually unavailable
      – Related, there's a need to 'think different' about servers; the advent of disposable IT (e.g. some times it's easier to throw away a broken server and re-launch rather than to 'fix' it)
      – Cloud storage will have nuances about how it is handled differently from typical IT storage infrastructure

      I'm sure there is more.

  5. Rob Minaglia says:

    Randy – What are some of the problems you see in BSS functions like metering, rating and billing, from both the the service provider and consumer perspective, for both models?

  6. randybias says:

    Rob, the key issue for BSS functions is multi-tenancy. Just like the telcos, perhaps more so, clouds will eventually need fairly sophisticated and robust mechanisms for metering and billing. We already have partners who desire to resell the platform while maintaining control of the customer relationship. For them it's important to have very good tools for managing metering/billing.

    I don't know if there is an inherent difference of approach in terms of what cloudcenters need to do differently from infrastructure web services. My guess is that there is not a significant difference there. Both need to find ways to allow not only clear transparent metering, but very granular (down to a 'user' or 'department') billing capabilities.

    Thanks for your great question.

    –Randy

  7. [...] opportunity to explain why GoGrid is different and about cloudcenters in particular, which are an interest of mine. Related posts:Pre-release CLI Utility for GoGridGoGrid Opens Up API [...]

  8. shu says:

    Hello,
    Thank you for an interesting article. Let me ask a question. Which part of a cloudcenter is virtualized? It seems a cloudcenter is "a traditional data center + server virtualization + NAS". My understanding is correct? or network infrastructures (e.g., firewalls) are also virtualized?

    • randybias says:

      Yes, effectively the entire cloudcenter is virtualized including loadbalancers, firewalls, and NAS. You will never see another customer's infrastructure nor will they see yours.

  9. [...] podcast, Overcast, interviews myself and Michael Sheehan.  In this latest episode, #6, we cover cloudcenters in detail.  I really enjoyed it, although I think there are a couple of editing errors in the [...]

  10. Wesley Stephens says:

    This is a great post! I have a couple of clarifying questions though:

    - "Unfortunately, every service above is based on an Amazon standard, not an industry standard."
    Amazon is delivering all its infrastructure services using the SOAP protocol. It is true that the parameter requirements and such are particular to Amazon as they represent Amazon's custom developed API, but from a client perspective all the services can be interrogated and invoked using an industry standard protocol. Did you mean something different here?

    -EC2 provides Xen hosting, but image management and storage is completely custom.
    Amazon uses a modified version of the XEN Hypervisor to effect their virtualization, and under the covers image management and storage management is custom, but it would not seem that none of that is visible or of real importance to a client of the services. Am I missing something here?

    -SQS does not use any standard queuing or messaging protocols such as JMS or STOMP.
    This one I get. Building an application based on Amazon's queuing system would clearly impact a client should they decide to relocate their application to another cloud provider

    -SimpleDB now has an ‘SQL-like’ interface, but is essentially a 100% ground up creation of Amazon.
    Ditto for this one – I think most enterprise folks will want to stick with a true SQL solution, not a non-portable look alike.

    So I guess, I really just have the two items I am seeking clarification on.

    Thanks, and again this was a great post!

    • randybias says:

      Wesley,

      Thank you for your questions. It's true that the Amazon APIs all have a SOAP interface. This is really not my point in talking about industry standards. To some degree any API demands that you use *something* that the industry uses or it won't be adopted. More important are services like S3, Amazon's Simple Storage Service. A 100% proprietary file storage system. You won't be able to have an equivalent in your own datacenter.

      With regards to EC2, the Xen hypervisor is standard, but that's about it. Storage for EC2 images is in S3, a proprietary standard. The image management and description system is also proprietary. Or, at the very least, you (again) can't have it inside your datacenter. Well, perhaps you can leverage EUCALYPTUS, but their legal status on the structure of their API interfaces is almost certainly in question. More so now that they are a commercial entity. Do you want to bet the farm that Amazon won't sue given their history with '1-click shopping'?

      There are two ways to look at whether a given service is proprietary or not:

      1) Can you get it today in your own datacenter, infrastructure, or cloud?
      2) If you write tools and software that manages the service through an API can you reuse those easily with other similar systems?

      My guess is that with most of Amazon it's 'no', 'yes, but it will be painful', or similar. Amazon could choose to provide a de facto standard, but until they take a stance with regards to the legality of copying their programming interfaces and services then sticking with well known standards seems like the best bet.

      Look more towards the efforts of the OGF around the OCCI standard, Sun's ZFS, and similar for real-world examples of standards or software that can be safely embraced inside and outside the cloud.

      –Randy

  11. [...] leads this space with their technology and their commitment to be the world’s #1 ‘cloudcenter‘ (still one of my most widely read GoGrid blog [...]

Leave a reply