The message below was sent to GoGrid customers from our CEO/Co-Founder, John Keagy, and CMO/Co-Founder, David Hecht, regarding the Distributed Denial of Service (DDoS) Attack which affected many GoGrid users.

It has been a long couple of days here at GoGrid. We are hearing from many of you that you want more information, more often. Engineers are often working so hard to fix an issue that you they don’t give you enough visibility into what is going on.

In that spirit of improved communication and transparency, I want to continue sending updates via email. You can always see the latest status at http://www.GoGridStatus.com as well.

While some of you have been unscathed by the network attacks and portal issues over the past few days, we know that many of you are frustrated at the downtime and impacts to your Internet infrastructure. We are frustrated as well. We’ve been in the hosting business for over 8 years now, and have generally been able to prevent most incidents from impacting customers as heavily as this attack did.

ONGOING DDoS ATTACK

Our network is currently the target of a large, distributed DDoS attack that began on Monday afternoon.   We took action all day yesterday to mitigate the impact of the attack, and its targets, so that we could restore service to GoGrid customers.  Things were stabilized by 4 PM PDT and most customer servers were back online, although some of you continued to experience intermittent loss in network connectivity.

We had a maintenance window scheduled for 9 PM PDT to do a major expansion of GoGrid’s capacity and roll out some minor feature improvements and bug fixes. Because this maintenance window required the portal being down and support cases would have to be opened by phone, we considered postponing the maintenance to a time when things were calmer.

In the end, the decision was made to proceed with the maintenance because this capacity expansion had been planned for several months and would give us more flexibility in ensuring low utilization across our infrastructure. In hindsight this may have been a poor decision because the maintenance took longer to complete and the maintenance window had to be expanded by several hours.

ROUTING ISSUES THIS MORNING

We spent the night cleaning up servers that were still down, reboots that did not happen properly, and other issues, and continued to develop plans to establish a long term solution to this ongoing issue.

Beginning early this AM PDT, our support team began to get more and more reports of servers that were unreachable from certain parts of the Internet. All of these servers were pingable and accessible from our testing connections outside the GoGrid network, but not to all locations worldwide. There appeared to be a routing issue with some networks not properly announcing GoGrid routes. Some of your web sites appeared offline to most or all of your own customers, while many were unaffected.

CURRENT STATUS

The routing issue was resolved around 11 AM PST today.  Our network engineers localized the problem to an issue with our border routers improperly announcing some routes.  The issue was resolved by clearing our BGP cache on our border routers.  We are not certain at this time the root cause of the issue, and are continuing to investigate and will provide an RFO soon to customers who opened Cases.  We suspect the issue had something to do with the changes we implemented in an emergency maintenance window, yesterday, as part of our efforts to mitigate the DDoS attack.

If you are continuing to see any connectivity issues with your GoGrid servers, we ask that you run a traceroute to your servers’ IP address so you can provide it to our support staff when logging an issue at http://my.GoGrid.com

We appreciate your patience during this difficult time, and thank you for being a GoGrid customer.

I personally would like to apologize for these issues as well. While out of our control for the most part, these DDoS attacks affected many of you and disrupted your livelihood, business and personal life.

It’s important to us to clarify GoGrid’s position with regard to cloud computing standards and the Open Cloud Manifesto (OCM). There has been a fair bit of controversy in the ‘blogosphere’ recently over the OCM, which is to be released on Monday.

In particular, myself and Steve Gillmor (of TechCrunch IT fame among others), had a somewhat heated, but friendly exchange over his scathing assessment of the situation. Steve invited me to a “News Gang” podcast of the Gillmor Gang on Friday, which was posted here. During that live podcast he asked us to clarify GoGrid’s position.

This post is really about making sure everyone is on the same page and understands how GoGrid views the OCM and cloud computing standards in general.

Background
It’s unnecessary to recap everything in detail. I think James Urquhart handled this fairly succinctly. Geva Perry also has a nice summary including a link to the draft document. In a nut:

1. Some folks tried to lay down some guiding principles for “open” cloud computing in the Open Cloud Manifesto
2. Some folks reacted badly feeling that the process wasn’t actually “open”
3. Bruhaha ensued

Who cares?

Well, we all should really. From our perspective this is a healthy, yet contentious debate. We think there were good points and missteps on all sides. In particular, we think it’s important to realize that given how interconnected we’re all becoming it’s actually very hard for any given group to monopolize the Internet, the “cloud”, or similar.

We believe the following to be true about what happened:

• The folks involved in the early Open Cloud Manifesto did not intend to “shut out” anyone
• The process around building the Open Cloud Manifesto could have been more ‘open’
• The manifesto is not about setting a standard, but starting a conversation
• This “conversation” is meant to be about principles that already apply to the Internet

Position on Open Cloud Manifesto
We continue to be an enthusiastic supporter of the Open Cloud Manifesto (OCM) and open cloud standards in general. That’s why we licensed our own API under an open license in January. The OCM is an important move forward in the emerging debate about what “Cloud” and “Cloud Computing” mean. We do not support any kind of exclusion in the OCM or of folks who want to be it’s supporters. We believe everyone needs to have a say in these guiding principles. In fact the OCM itself is largely about saying how much “The Cloud” needs to be open, unfettered, and democratized.

That means everyone needs to be involved.

Summary
Simply put: contentious conversations, vibrant arguments, and great people will all eventually yield the right results. We don’t think it’s possible for anyone to cordon off and monopolize this conversation, foist standards on others, and won’t support such efforts. And, we don’t think the Open Cloud Manifesto is anything but a well-intentioned attempt to move the conversation forward. One that was never meant to be ‘closed’ and come Monday when it’s officially released we’ll see that it’s a positive move and all of the folks who worked together on the OCM (including Reuven Cohen, IBM, and many others) should be commended for their attempts to get everyone on the same page even in the face of extreme controversy.

–Randy

Over the past year, I have written about the various primal layers of Cloud Computing. Typically, my role is to “over simplify” in order to make the Cloud a bit more palpable by “the masses.” My colleague, Randy Bias, is the resident über-tech, so I usually leave the more complicated developer and sys-admin posts to him. As we all know, the Cloud is hot and becoming increasingly complicated as new products, services and vendors throw their hats into the ring. But is this over-complication confusing and saturating the market? I think not, in terms of the latter, but it is truly becoming more confusing.

First, we at GoGrid, broadly define Cloud Computing as such (latest definition):

On-demand self-service Internet infrastructure where you pay-as-you-go and use-only what you need, all managed by a browser, application or API.

Even that definition I feel is a bit skewed toward Infrastructure. Probably more aptly defined, it would be:

On-demand, self-service Applications, Platforms, Services or Infrastructure dynamically consumed on a pay-as-you-go basis using a browser, application or API.

Definitions evolve and morph over time. This is probably the 30th iteration of our definition over the past year.

So I will circle back to the Cloud Pyramid (as seen below):

To briefly recap the different layers:

• Cloud Applications – many view this layer as containing SaaS (Software as a Service). It’s important to remember that not all SaaS offerings fall into this category. SaaS existed well before the term “Cloud” came into play. Essentially, the idea is that Application functionality is served via the internet and this application typically does one thing. This could be email (e.g., Gmail) or CRM (e.g., SalesForce).
  • Advantages – available via a web browser, rich interfaces, frequently free or paid either by monthly usage or seat licenses
  • Disadvantages – little or no customization available, limited to the feature-set provided
• Cloud Platforms – otherwise known as PaaS (Platform as a Service). Typically a development language or framework (e.g., Ruby on Rails, Python, .NET, Java) is contained within this environment. What this means is that users consume the hosted framework. Examples are EngineYard (a RoR  stack hosting environment), Google App Engine (supporting the Python framework), Microsoft Azure (running .NET framework) and Force.com (proprietary SalesForce.com framework) for example.
  • Advantages – the frameworks are hosted by vendors. This means that the underlying infrastructure is controlled, updated and managed by Cloud Platform vendors.
  • Disadvantages – while offering significant more control over the development environment, because the underlying infrastructure is not available to the end-developer, these developers are “at the mercy” of the hosting provider to ensure updates and management of the various framework stacks are fully functional, updated and accurate.
• Cloud Infrastructure – this is called IaaS (Infrastructure as a Service). At the lowest layer of the Cloud Pyramid, infrastructure is delivered and consumed on-demand utilizing some sort of paravirtualization and/or hardware integration. This layer includes servers, networks and other hardware appliances (e.g., load balancers) delivered as either Infrastructure Web Services (e.g., Amazon Web Services) or as “cloudcenters” (e.g., GoGrid). More information about the differentiation we make between Infrastructure Web Services and Cloudcenters is discussed in the posts here.
  • Advantages – full control over the various components of infrastructure means that you can work with the infrastructure in just about any way you desire. It also lays a fundamental groundwork for building other Clouds on top of it (especially Cloud Applications)
  • Disadvantages – sometimes more expensive compared to the other layers; if you aren’t familiar with full access to infrastructure, controlling and managing could be daunting.
• Other Cloud Services – there are many types of ancillary Clouds that are showing up including: Cloud Services, Cloud Storage, Cloud DB, Cloud Aggregators, Cloud Extenders, Cloud Management, etc.

Obviously, this just scraped the surface of the Cloud. But let’s take a quick look at one particular “application” which can span all layers of the Cloud.

Microsoft Exchange is probably something that many of you are familiar with. For the uninitiated, Microsoft Exchange is a messaging and collaboration application developed by Microsoft and is contained within its line of server products. Functionality includes email, calendaring, contact management and tasks. Why have I chosen Exchange as a good example to use for traversing the various layers of the Cloud? Simply because some form of Exchange can conceivably exist at each layer. Let’s explore Exchange on the…

• Cloud Application layer – you want all of the functionality of an Exchange account but don’t want to worry about the management thereof. At this level, you get just that, the ability to have a hosted Exchange mailbox with many of the bells and whistles of a standard corporate Exchange account but without the fuss. Billing for this is typically by user by month, typical of many SaaS or Cloud Applications.
• Cloud Platform layer – suppose your corporation outgrows a simply leasing of individual Exchange mailboxes as present within the Application layer, you can then opt for a solution of a dedicated Exchange server, hosted by a provider. While more of a dedicated play, conceptually, the idea is the same. You choose a hosting provider who manages the infrastructure (the experts) and “frameworks” and you simply administer the usage and functionality therein. Providers integrate other functionality (e.g., web access) into the product offering while still protecting the underlying infrastructure. Companies have more control at this layer.
• Cloud Infrastructure layer – assuming that your company has grown to the point where you need a more robust corporate infrastructure, you probably are looking at setting up a clustered Exchange environment. At this layer, you would need to have full access and control over your infrastructure in order to set up Active Directory and other protocols. Hosting with dedicated, the cloud or a hybrid solution (e.g., using Cloud Connect)  is the best implementation here.

With the Cloud, you can grow your infrastructure based on the demand and needs of your company. The Microsoft Exchange example can just as easily be applied to moving your Web Application through the different layers of the cloud as well, for example a CRM application.

While this is not exactly a true “Cloud play,” I hope that it helps to explain how the layers of the Cloud Pyramid are differentiated in terms of control, scalability and functionality. You could also use an example of Ruby on Rails or Python (at least for the bottom two layers). With Google App Engine or EngineYard, you work within the languages and frameworks available. If you move down to the Infrastructure layer, you can do that as well, but you also have the ability to control the underlying infrastructure and customize the framework environments to your liking. Unfortunately, I’m a bit hard pressed to explain how frameworks can be utilized at the Cloud Application level, but I’m open to other comparisons or examples.

What other applications or environments can traverse the Cloud Pyramid? I’m sure there are many!

If you are currently a GoGrid customer, I want to talk to you! As you know, the Cloud is all the hype with lots of vendors, analysts and industry experts expounding the virtues and benefits. (I’m one of those who keeps the grease in the marketing buzz machine rolling.) But I’m also an advocate, not only of Cloud Computing, but especially of those using GoGrid successfully.

So, here is what I’m looking for…

• Windows Developers – people using the “Windows-side” of GoGrid. That is to say, ASP.NET, Windows Server 2008 or 2003, Microsoft SQL Server 2005/2008, IIS
• Linux Developers – those implementing either Red Hat Enterprise Linux and/or CentOS
• OpenSource Users – if anyone is developing rich applications using PHP, mySQL, PostgreSQL, Apache
• Mixed Environments – of particular interest to me is those using combinations of the items above
• Failover Scenarios – are you using GoGrid as a failover site? For redundancy?
• Cloud Connect – have you created a hybrid hosting solution that you are particularly proud of, using a combination of Cloud and Dedicate hosting?
• Twitter Development – since Twitter is all the rage, are you creating the next killer app to manage the Twittersphere?
• Statistical Analysis – are you spinning up a series of high end GoGrid servers for a few hours to do number or other data crunching?
• Web 2.0/3.0 Products – how are you AJAX-ing your applications using GoGrid for hosting? What great new mashup application have you created?
• API Activity – are you using the GoGrid API in any particularly exciting way? How ARE you using the API?
• Cloud Storage – how have you mixed in GoGrid’s Cloud Storage into your infrastructure? As a backup solution? To server data?
• Digital Media Manipulation – are you using GoGrid to process large sets of digital media?
• Database intensive products – have you implemented an architecture that utilizes a high-end DB either in the GoGrid Cloud or cloud-connected to a dedicated server?

So, what do you get out of it? I can definitely promise a few things:

• Free PR and Marketing – tell us all about the product or service that you are proud of and we will write about it,  include in our newsletters and pass around various social networking sites
• Web links – we will include your site in links on our sites and blogs
• Potential presence at trade shows – if you are chosen as a showcase customer, we welcome you at our upcoming tradeshows
• Potential participation in our Referral programs – please contact me for details
• Other forms of “Thank you’s” – (wink)

To submit your information, please complete the form below:

cforms contact form by delicious:days

I look forward to hearing about and trying your sites that run on GoGrid. Over the next few weeks, I will be posting up some write-ups of your submissions.

San Francisco PostgreSQL User Group’s (SFPUG) April Meetup will be held at the GoGrid offices, specifically discussing how GoGrid can be used as a Cloud hosting infrastructure for PostgreSQL applications. Space for this event is limited to 30 people (15 have already been filled as of this writing).

Location: GoGrid offices in San Francisco
Date: 4/8/09
Time: 7:00pm
RSVP: RSVP’s can be made via this site: http://postgresql.meetup.com/1/calendar/9936194/

The event will be streamed live as well. Food will be provided.

Please act now as space is limited.