We absolutely LOVE hearing how GoGrid customers are using our cloud solutions to create unique “cloud fingerprints” and environments using the features and data centers of GoGrid. Paul Trippett just published a very interesting write-up of an infrastructure environment that addresses many of the common concerns facing any company looking to provide a highly-redundant infrastructure while also ensuring a solid Service Level Agreement (SLA) for their customers.

You can find Paul’s original write-up titled “Utilizing GoGrid’s Multiple Data Centers for Routing and Failover” on his site. With his permission, we have reposted the article so that others can learn, mimic and build upon his unique scenario.

At the beginning of the year one of our customers asked us if we can provide an SLA for StormRETS and with it, the sound gritting teeth suddenly echoed around the room. As you can imagine, this caused more questions than which we actually had answers for:

What kind of SLA did we want to provide and what could we realistically provide?

Our hosting provider, at the time, had an SLA which entailed “We don’t give any guarantee that your servers will be available, but if for any reason they are unavailable we will get the back up and running as soon as we can.”, erm, how on earth can we build a SLA based on that. It was decided at this time we would migrate our servers to another hosting provider, one at least with a SLA we can build on and a company we can actually contact directly should a problem arise.

Any migrations we did needed to address the following issues:

• System Monitoring — Constant monitoring for potential problems.
• Network Downtime — If its a localized network issue, secondary DR fail-over site.
• Server Downtime — Multiple Servers or Secondary DR fail-over site.
• Software foul ups — Unit Testing and run the system on multiple servers.
• Server Software Updates — Multiple Servers.
• Maintenance Schedules — Multiple Servers.
• High Traffic Spikes –  Multiple Servers.
• Botched Deploys — DR fail-over.
• Drunk SysAdmins

We decided it would be a good idea to setup a second data center in the case of a failure of the primary, but, what’s the point in having all this extra duplicated capacity we’re not using, it makes sense to put it to good use by directing traffic to our nearest data center and failing all traffic over in case of a major outage.

DNS Failover and Load Balancing
The problem with this is that DNS fail-over or geo-aware DNS was extremely expensive. We just couldn’t justify a spend of more than $200 a month for something we could setup ourselves on a few VPS boxes scattered around the globe for $50 a month. Anycast DNS is severely overrated, it makes sense yes but not at the prices being asked. Sometimes answers comes from strange places, while doing some whois searches on start-up companies, which we knew would be looking at this same problem, we found Zerigo who have recently started offering geo-aware DNS at prices starting at $20 per month. After running some tests there response times aren’t to shabby either!

Cloud Hosting
There are more than enough blog posts about choosing a cloud provider. We looked at the more common providers including Amazon, Rackspace and GoGrid, in the end we decided on GoGrid. GoGrid offer a really good SLA, they have telephone support and multiple data centers ready for you to use.

GoGrid offer a really good SLA, they have telephone support and multiple data centers ready for you to use.

With every GoGrid account comes two /28 blocks of routable IP addresses, one in each data center. This is an awesome feature, usually when you create a new VPS you are assigned a random IP address from a huge pool, deleting a VPS would mean you lost that IP address, with GoGrid you can delete a VPS and re-assign its IP address to another VPS meaning all your IP addresses are always contiguous and easy to remember. Due to GoGrid’s network setup you cant use all the IP addresses as some are reserved for the default gateway, network broadcast, and a further 3 IP address reserved in the middle of the pool for the active and standby load balancers.

GoGrid provides free fault tolerant F5 load balancers with their service, allowing you to setup up to 3 load balanced IP addresses per data center. In our old setup we had setup Load balancers ourselves running on CentOS and NGINX but using GoGrid for this saves us time and money, and gives us one less thing we have to worry about and manage ourselves.

Network Setup
Our network setup is nothing new, but the data we have, needs to be retrieved and processed quickly via our API. Our average API response time is currently 0.07 seconds against approx 1,000,000 property records and we don’t want any redundancy we put in place to affect that time. Additional locations should be as autonomous as possible with little or no inter-site communication caused by an API request and be able to handle another data center going down.

Above you can see the basic network diagram of what we have setup and running:

• Zerigo DNS to load balance between the two data centers and fail-over requests to another data-center in case of a failure.
• GoGrid f5 load balancers in each location to load balance requests across the web servers in each location.
• OpenVPN Servers to bridge the two networks for passing replication data between the data centers.
• MySQL circular replication between the two sites.
• CouchDB multi master replication between the two sites.

A lot of the fail-over is left up to the platform to decide. Every minute the monitoring system calls into the system via a URL, this script checks a few key things, such as MySQL and CouchDB availability, if any of these tests fail the script returns a failure status and the DNS automatically switches. We use 3 minute TTL’s on the majority of our DNS records, so in theory fail-over should take no longer than 3-5 minutes to complete.

With our new setup we can now redirect traffic to our second data center while performing maintenance on the other, and we are in a much better position to provide an SLA to our customers, but even after these major first steps we are still not in a position to provide an SLA quite yet. Over the coming weeks we will be running various performance tests and fail-over testing to verify that both data centers will be able to work independently of the other, based on these performance tests we will be able to devise how much capacity we have and at which points we need to start considering upgrades and adding capacity.

Do you have an environment running on GoGrid that you think is unique, helpful for others to see and learn from or are particularly proud of? If so, drop me an email: michael [at] GoGrid.com.

Last week we announced several updates to GoGrid’s cloud infrastructure services. One of the highlights of this update was the enhancement we made to the saving your GoGrid Server Images (MyGSI) process.

Up till now, saving your MyGSI has been a fairly simple 3-step process that was helpful for horizontal scaling, re-imaging of servers, disaster recovery, or setting up failover environments. Through the update, this process can now be completed in a single step.

We’ve created several helpful tutorials to help educate users on the new process. Please reference our video and text tutorials below or take a look at the MyGSI User Manual on the GoGrid wiki for more information regarding the new process.

Video Tutorial

Below is a video screencast on how to create a GoGrid Custom Server Image (MyGSI). (Direct YouTube link)

Text Tutorial

To create a GoGrid Server Image (MyGSI), the process is very simple and easy to do. First, within the Grid View, click on the server you want to create a Server Image from. When you do that, a Save icon will appear (see the green arrow below):

The Save button is also in the List view:

Once you choose the server that you want to save and click the Save icon or link, you will get the following prompt:

Note that if your server has more than 50 GB of data on it, the saving process will fail and you will need to reduce the amount of data on your server.

Simply give your server image a name, description, and indicate whether it is a Web or Database server.

After this, you server will be gracefully shut down, and the image saving process will start. You will see your server’s status light turn to amber:

And there will be an entry in the Jobs tab that not only states that the action has started, but also after a few seconds, displays the expected time for the server to be fully saved to cloud storage. If there is no “Estimated Job Duration,” just wait a minute and refresh the Jobs tab.

Your server will come back on line shortly.

Once the server image is created, you have the ability to instantiate new servers from it, or share it with the GoGrid community as a Community GSI (CGSI).

Lastly, the MyGSI process is backwards compatible, meaning that if you had created a Sandbox Server previously and it still appears within your GoGrid account, you can still convert it into a MyGSI using the same older process. Also, any MyGSIs created using the legacy process will still be available to create new server instances. However, the creation of Sandbox Images has been removed as of this release, and replaced with the much easier process described above.

Have any questions about the MyGSI update? Contact our support team. We’re excited to help you!

Part of a new Engineering and Product initiative at GoGrid is the fact that we will be providing more frequent updates to our Cloud Infrastructure Services. These updates may take the form of new or updated features, bug fixes, or back-end enhancements. Our goal is to provide these updates and feature releases on a monthly basis (more information on our new agile software engineering process will be coming) so that we can answer customer requests and needs on an extremely timely basis as well as introduce revolutionary services to the Cloud Infrastructure marketplace.

Today (March 11, 2011), we released our March 2011 update (code name “Long Beach”). The major features and updates are listed below:

• Enhancement to the MyGSI Feature
• Usage Report
• Billing Improvements
• Ubuntu 10.04 Cloud Server Support
• Debian 5.0 Cloud Server Support

Please read on for details on each of the items listed above.

MyGSI Updates

About a year ago, we introduced a new feature called MyGSIs – customized GoGrid Server Images – and how easy it was to create and save a Server Image. Server images are helpful for horizontal scaling, re-imaging of servers, disaster recovery, or setting up failover environments. Until this update, we had a pretty straight forward 3-step process: 1) create a Sandbox Image, 2) configure your server, prep it, and run scripts, and 3) save the server to cloud storage.

Now, we have consolidated this process into ONE step. Simply select the existing server you want to save and click the Save icon. That’s it! There is no need to create a Sandbox Image, run preparation scripts or anything else. And, the best thing is, any existing server can be saved as a server image.

The Save button is also in the List view:

Once you choose the server that you want to save and click the Save icon or link, you will get the following prompt.

Note that if your server has more than 50 GB of data on it, the saving process will fail and you will need to reduce the amount of data on your server.

Simply give your server image a name, description, and indicate whether it is a Web or Database server.

After this, you server will be gracefully shut down, and the image saving process will start. You will see your server’s status light turn to amber:

And there will be an entry in the Jobs tab that not only states that the action has started, but also after a few seconds, displays the expected time for the server to be fully saved to cloud storage. If there is no “Estimated Job Duration,” just wait a minute and refresh the Jobs tab.

Your server will come back on line shortly.

Once the server image is created, you have the ability to instantiate new servers from it, or share it with the GoGrid community as a Community GSI (CGSI).

Lastly, the MyGSI process is backwards compatible, meaning that if you had created a Sandbox Server previously and it still appears within your GoGrid account, you can still convert it into a MyGSI using the same older process. Also, any MyGSIs created using the legacy process will still be available to create new server instances. However, the creation of Sandbox Images has been removed as of this release, and replaced with the much easier process described above.

Be sure to read the MyGSI User Documentation that can be found on the GoGrid Wiki.

USAGE REPORT

Included with the “Long Beach” update is a new feature to help you get a better view into the activity within your GoGrid account. There is now a new tab called “Usage” which is under the My Account section. This new report is in beta for the next 90 days.

The Usage Report overview is broken up into two sections: Metered Services and Fixed Services. Metered services are things like RAM, Storage, and Outbound Transfer. Fixed services are for items that recur regularly during a month’s time, things like monthly licenses or IP addresses allocated to your account. Metered services are more variable in nature.

The Usage Report shows a quick snapshot of Metered and Fixed Services for the current or previous billing period. NOTE:  The historical data in this report will not be backfilled; therefore the previous billing period will not be viewable as of this release. Also, the Usage data collection officially starts on March 11th, 2011. This means that the current billing period report may be inaccurate for up to 30 days for some customers. Once a full billing cycle passes, the Usage Report will show accurate data.

Under the Usage Menu, you can see a link to “Download usage report”. After selecting the billing period from the drop-down menu, click the link to download all of the associated usage data for that period and it will be downloaded to a CSV file.

The CSV file contains the usage statistics for the following objects:

• Cloud Servers – # of GB Hours
• Outbound Transfer – # of GBs
• Licenses – # of instances of a particular license
• Cloud Storage – maximum GBs stored for that day
• Load Balancing – # of instances of a load balancer
• Public IPs – total # of IP addresses allocated to the account
• Operating Systems – # of instances of a particular Operating System in use

The fields reported are listed below:

• Account Name
• Customer ID
• Report Start date/time
• Report End date/time
• Object Type (see above)
• Name of the Object
• Description of the Object
• Data Center
• Usage Start date/time
• Usage End date/time
• Quantity
• Units

A quick tip, be sure to format the “reportStart,” “reportEnd,” “usageStart” and “usageEnd” to mm/dd/yy hh:mm to see the full details with the date and time.

For additional details, visit the GoGrid Customer Portal Guide that discusses the new Usage feature.

Billing Improvements

One of the advantages of utilizing GoGrid’s Cloud Infrastructure Services is the ability to choose a Pay-As-You-Go plan or a Volume pricing plan. We are allotting more focus to the “billing experience.” In the past, our focus was on the delivery of the services and after engaging with our customers, we realize that some time and care was needed for the overall experience. To that end, we have kicked off a complete overhaul of the billing process to provide you with not only a better experience but also with more insight into your account usage.

This first iteration of Billing Improvements consists of the following changes:

1. Monthly Invoices
2. Account Plan Changes
3. Usage

The details of each of these items are listed below.

Monthly Invoices

To make the billing and invoicing process easier, we are removing daily invoicing. From this point forward, customers will now receive monthly invoices on their monthly bill date. The monthly bill date is the date on which a customer opened their account with GoGrid.

There are a few exceptions in which additional invoices may be generated:

• Fixed Services – if you are using a fixed service like dedicated servers, VPN, or hardware firewalls, you will receive an immediate, one-time invoice. The charge(s) for this service(s) will be pro-rated through the next bill date. However, the next monthly invoice will include a full charge for the next month’s fixed service and is paid in advance.
• Manual invoices – if there are manual invoices generated off-cycle.

NOTE: There may be a bit of a transitional period during March 2011 because of the switch from daily to monthly invoicing. If you have any questions, please contact your account manager.

Account Plan Changes

We have updated the process of making changes to your GoGrid account plan and have a clear trail of the changes you have made and when they were made. We have also added email notifications to the process so that you get two email confirmations: one when you request a change and one when the change is implemented.

Just as you could in the past, select either the RAM or Data Transfer Plan and then choose whether you want your plan to be Monthly or Annual (you get a better discount if you choose an Annual plan):

Confirm the plan you want and accept the terms of service.

Once you make your selection, there will be an indication of the plan change within the Plan Selection page (in red) and your current or active plan will appear grayed out:

Also, within the Jobs tab, you will see an entry confirming your plan change.

You may upgrade or downgrade your various plans as you see fit. Downgrades take place on the customer’s next bill date. Plan Upgrades take place retroactively back to the beginning of the current billing period.

The process for Upgrades is a bit more complicated. On the next bill date, customers are credited the last month’s lower plan charge and are charged for the new plan for the previous month and in advance for the next month.

Sometimes examples are a bit easier to understand:

In the case above, you can see how the charges for the lower plan are reversed out and the new plan is retroactively applied.

In most cases, people upgrade plans during a current month because they are going into an overage situation or in anticipation of incurring overage charges.

Ubuntu 10.04 Cloud Server Support

With this release, we are pleased to announce that Ubuntu 10.04 32- and 64-bit Cloud Server Operating Systems are now available to complement Ubuntu within the Dedicated Server Operating System choices. This is the LTS – Long Term Support – version of Ubuntu.

Debian 5.0 Cloud Server Support

Also with this release, we have included Debian 5.0 32- and 64-bit Cloud Server Operating Systems to complement Debian within the Dedicated Server Operating System choices.

More to Come

The “Long Beach” release is the kick-off of a new agile development process here at GoGrid. We will be aiming to have regular monthly releases from this day forward so stay tuned for more exciting news from GoGrid. Be sure to subscribe to the GoGrid Blog Feed to get regular updates.

Last week GoGrid CEO, John Keagy, was a guest on “This Week in Cloud Computing” hosted by Amanda Coolong & David Linthicum. The video podcast covered a variety of interesting topics including Fujitsu’s new cloud offering in Asia-Pac, Microsoft’s war of words with Google over Cloud Connect and Intel’s rumored client-aware cloud offerings. John Keagy weighs in with some very interesting thoughts on each of these topics. We’ve embedded the full episode for your viewing pleasure, but we’ve also included highlights and the discussion about GoGrid’s past, present and future!

Note: each of the clips below the main one will jump directly to the relevant content.

Full Episode

Highlights

Australia is the first country outside of Japan to roll out Fujitsu’s standardized cloud offering. Do you think the expansion of cloud computing in Asia-Pac will add a boom to business development?

Google and Microsoft are sparring over Cloud Connect and Microsoft Office. Microsoft claims that Cloud Connect is Google’s ploy to get their hands on more of our data. Should Microsoft be worried and is Google going to succeed with enterprise?

Intel is planning a Cloud Vision event and are expected to reveal client-aware cloud. Will this mean cloud providers will write for devices directly and move further away from the browser and closer to apps?

Amazon launches AWS Cloud Formation to make easier for enterprises to mack stacks of apps and recourses. With this ability to combine resources, will this provide a strategic advantage for Amazon?

Amanda Coolong, David Linthicum and CEO John Keagy discuss GoGrid’s past, present and future.

Do you have an opinion on what’s happening in cloud computing? Share in the comments section below!