Recently, GoGrid was examining performance enhancements on several internal processes; among these enhancements was switching from standard gzip to “pigz”. Since I had never heard of this “pigz”, I was intrigued by this supposed “parallel” implementation of gzip; meaning it uses all available CPU’s/cores unlike gzip. This prompted me to ask, “I wonder if there is a parallel implementation of bzip2 as well”, and there began my endeavor.

pigz and pbzip2 are multi-threaded (SMP) implementations of their respective idol file compressors. They are both actively maintained and are fully compatible with all current bzip2 and gzip archives.

If you’re like me, you might’ve stayed away from using gzip or bzip2 due to the single-threaded aspect. If I try to compress a, let’s say, 2GB file, the system becomes rather sluggish; the reason being is that the “compression tool of choice” uses almost all of 1 core of today’s multi-core, multi-CPU systems and creates an uneven load between the cores, causing the CPU to operate very inefficiently.

In this example I have a .tar file with several databases in it, which totals 1.3GB. The system in question is a GoGrid dedicated server with 8 cores. The server’s load is around 1 and is a production database server.

Using bzip2, the file took approximately 6 minutes and 30 seconds to compress. Yikes!

Now we’ll try this again with pbzip2, the parallel implementation of bzip2.

Not only did pbzip2 take roughly 1/7th the time as regular bzip2, it has the verbose option that provides some nice output and a progress bar (not visible here) while compressing.

The file compressed to an impressive 127M down from 1.3GB using bzip2 or pbzip2

Now let’s try the same test with gzip2 and pigz.

# time gzip dbbackup-12_10_2011.tar

gzip took a considerably less amount of time than bzip2 to compress the same archive; roughly 40 seconds instead of 6 and 1/2 minutes. However, the resulting file is a bit bigger at 177M.

Now with pigz.

pigz took about 1/7th the time as gzip did and clocked in around 6 seconds; an impressive speed again.

Conclusion

Both parallel implementations provided a very large time difference in comparison to the standard, single-threaded implementations of these compression tools.

pbzip2 and pigz provided an almost spot-on 7/8ths time difference between using 1 and all 8 CPU cores — an impressive performance gain. pbzip2 and bzip2 also compress the files a bit better than gzip – %93 compression ratio.

In particular, this could save you a lot of money when using GoGrid Cloud Storage, as well as save your Cloud or Dedicated resources and stability for the purposes they were intended for in the first place.

Depending on how these archives will be used, you may consider either implementation based on your needs. pbzip2 has a higher compression percentage, while pigz is faster.

Whatever you choose, hopefully this will increase the usability of compression tools and help provide a more stable and optimized environment GoGrid environment.

Happy Compressing!

This is the 3^rd and final post in my setup and use of the GoGrid Community GSI server for Cacti Monitoring. In my first post, “Set Up A Cacti Monitoring Server in Minutes with this GoGrid Community Server Image,,” I covered how to deploy Cacti in your GoGrid environment using a Community GSI. My second post, “How to Monitor Your Ubuntu Server on GoGrid in 6 Steps Using Cacti 0.8.7g,” I discussed how to initiate monitoring of your GoGrid Ubuntu server. Now to round things off, I want to show you how to link up your Cacti monitoring server to a Windows Server 2008 server on your GoGrid network. The base install of Cacti 0.8.7g will allow you to monitor the server’s bandwidth utilization, Ethernet errors, number of logged in users, and total number of processes. There are other templates available to monitor other components and services on your Windows server, but they require using an additional SNMP service beyond the Microsoft SNMP service. My blog post won’t get into the latter, but I will cover the former.

Objectives:

1. Configure GoGrid private network connectivity on Windows 2008 Server and test connectivity to Cacti server
2. Configure and start Microsoft SNMP service on your Windows 2008 Server
3. Add new Cacti device
4. Create graphs to log Local Connection and Local Connection 2 bandwidth and errors, Logged in Users, and server processes

Configure GoGrid private network connectivity on Windows 2008 Server and test connectivity to Cacti server

Below we see that we have a server (“Web2”) deployed on GoGrid with a public IP. Let’s log into this server and configure the private network with a private IP from the same subnet of the Cacti Monitor server. As I described in my previous post – I am using the prescribed private IP subnet from my GoGrid portal, contained under the List tab and then under Network – Private Network.

Once logged into the Windows 2008 server (“Web2”), I go to the Network and Sharing Center which is found by first going to the Start button. From here I need to open up Local Area Connection 2. This is the private network interface that plugs into your own private VLAN on GoGrid. I enter the “Properties” button and then open up “Internet Protocol Version 4 (TCP/IPv4)”. By default GoGrid will enable DHCP for this private network interface. If you have a DHCP server, your server can receive private IP addresses upon initial boot up. Perhaps I will cover this in a different post. For now, we need to configure the interface with a static IP available from our example private subnet 10.129.151.0/24. The subnet will be randomly generated for each customer account. I show you how to find this in my previous post, “How to Monitor Your Ubuntu Server on GoGrid in 6 Steps Using Cacti 0.8.7g.” Following the nomenclature – keeping the last octet of the 32bit address space the same as the public to make IP address association easier – I give this system the private IP address 10.129.151.110. The .110 matches up with the last 3 digits of the public IP address 173.204.58.110. This isn’t necessary, but helps you identify systems and know their private IP address based upon their public address and vice versa. See the image below of me assigning the private IP address.

Now, open up a command prompt >Go to Start, then Run – type in cmd and hit “enter”. I now test connectivity to the 10.129.151.0/24 network by pinging the gateway IP – 10.129.151.1. I get a response, so I can move on. If you don’t get a response, double check the private IP address you used and make sure the Subnet mask is correct. You can also test connectivity by pinging the private IP address of your Cacti Monitoring server. Remember, your 10.x.x.x/24 network will be different than my example subnet. Please check your portal for what you should be using.

Configure and start Microsoft SNMP service on your Windows 2008 Server

Check to see if you have the SNMP Service running already on your Windows system. By default, this server role in Windows Server 2008 on GoGrid isn’t installed and running. Go to Start -> Administrative Tools -> Services. SNMP Trap may be there, but we also need SNMP Service. To install SNMP Service go to Start -> Administrative Tools -> Server Manager -> Select Features -> Add Features. From here find and check SNMP Services (which covers SNMP Service and SNMP WMI Provider). See image below.

Click Next and this will begin the install process of this feature on Windows.

Once the SNMP Service is installed, we can go to the Services page to find and configure the SNMP Service with the appropriate community and host IP address to accept SNMP calls from.

First go to the Services page – Start -> Administrative Tools -> Services or from Run type services.msc

Find the local service – SNMP Service and right click it and go to properties. From here you need to give the service the community string that you will set on the Cacti server and the private IP address of your Cacti Monitor server. This is under Accepted Community Names and Accept SNMP packets from these hosts. See image below:

Add new Cacti device

This step is same “create a device” step that I outlined in my previous post – except the details of the host will be different.

The IP, hostname and template used in the screen shot below represent my example Windows 2008 server named “Web2.” I chose the Host Template Windows 2000/XP Host –> selected SNMP v2 -> put in the community string I chose, and clicked “create.”

You should see the SNMP information for the host quickly appears near the upper left portion of the screen. If you see an error here, you will need to check your private network connectivity between the two servers and check the SNMP Service configuration on the Windows 2008 server.

With the new device in place on Cacti, we can now create the graphs.

Create graphs to log Local Connection and Local Connection 2 bandwidth and errors, Logged in Users, and server processes

From the device we just created, go to Create Graphs for this Host.

From this page we want to add a check to the following graph templates seen below in the image:

1. Logged in Users
2. Processes
3. Local Connection
4. Local Connection 2

Once you have done this, click Create at the bottom of the page.

We will do this again for the In/Out Errors/Discarded Packets option next.

1. Change the “Select a graph type:” near the bottom of the Create Graph page to In/Out Errors/Discarded Packets
2. Next check box the “Local Area Connection” and “Local Area Connection 2”
3. Finally click the Create button at the bottom.

After about 5 minutes, the graph icons will be available and your data will then begin to accumulate for your viewing.

I hope this blog series was helpful for you. The GoGrid Cacti Monitor – Community GSI is a great server-based application that can easily be deployed to a GoGrid virtual server, and configured to communicate via SNMP with your servers on the GoGrid network. The information gathered will give you real-time and historical interface bandwidth, server performance, and other important system level information.

Be sure to check out other Community or Partner Server Images available on GoGrid. The GoGrid Exchange has many pre-configured software solutions that can be deployed to your GoGrid architecture in a matter of minutes.

In order to provide a more “rounded” voice on the GoGrid blog, we are going to start having some new authors. To kick off this initiative, I wanted to introduce Paul Lappas, GoGrid’s VP of Engineering and Co-Founder. Paul manages GoGrid’s engineering efforts, technical operations, IT and the technology vision for GoGrid.

Recently, he and some other team members attending a MeetUp in San Francisco at the Twitter headquarters to discuss and present  JClouds and Pallet and how those tools are being used at GoGrid. Here is Paul’s synopsis of Jclouds and the presentation:

“GoGrid is doing some really cool stuff using an automated provisioning technology called “Pallet”.  Pallet is similar to existing automating configuration technologies like Puppet and Chef but with a key difference that it was built specifically to solve the problem of quickly spinning up and configuring groups of servers in the cloud. It support “Jclouds” out of the box and is implemented as a set of libraries for “Clojure” which is a LISP-based programming language that is quickly building steam.  Jclouds is an open source framework developed by Adrian Cole that helps you get started in the cloud and reuse your java development skills, with an API that allows you the freedom to use portable abstractions or cloud-specific features.

GoGrid is working with the author of Pallet (Hugo Duncan) and a key contributor Toni Batchelli to enable the fast deployment of fully functional GoGrid environments for use by development teams for test & dev. It’s a tough problem for most companies, but especially challenging for us considering how complex (and capital intensive) it is to stage an end-to-end GoGrid environment due to the sheer breadth of technologies that span almost all 7 layers of the OSI stack. With Pallet, we are able to treat our “infrastructure as code” and manage the configuration of systems, networks and applications just like we do our source code so that they can be quickly applied to spin-up new environments. But perhaps the coolest aspect is that we are using GoGrid internally to virtualize the individual components! It’s kind of like Inception where there is a grid-within-a-grid. Our teams are still getting their heads around logging into a GoGrid account and seeing virtual representation of physical GoGrid components represented as VM icons in the GUI! Very cool stuff.

The following presentation provides more details of the implementation and was presented at the recent Jclouds meetup at Twitter’s headquarter in San Francisco. Check out future jclouds meetups here:
http://www.meetup.com/jclouds/”

Below is the presentation that was presented at the Jclouds meetup:

Stay tuned for more posts from some of the thought-leaders driving GoGrid.

GoGrid has long been supporter of open standards and the open source movement. We recently promoted how many open source solution providers are leveraging GoGrid as a platform to disseminate their open source solutions and technology in the cloud. Long before that at the beginning of 2009, we made our public API available to the public under a Creative Commons Attribution Share Alike License. This meant that under this licensing, developers could “share, distribute, display and perform the work” as well as “make derivative works” based on our API.

Now we are taking it one step further because we realize in order for Cloud Computing to be adopted across the business IT landscape, there needs to be more standards put in place as well as a commonality across the board when it comes to Application Programming Interfaces (APIs). The Deltacloud initiative is grabbing this issue by the horns by working to combine interoperability with an open sourced API. And, they are the only major cloud framework that isn’t leveraging a single company’s proprietary code, API or intellectual property.

Deltacloud, an Apache Software Foundation incubator project that provides an open set of application programming interfaces (APIs) to enable computing interoperability across private and public clouds, recently announced support for GoGrid and we are pleased to participate in this important initiative. There are quite a few advantages of this effort, specifically:

• The API can be offered by a cloud provider or individual user running their own server
• Multiple client libraries can be written in a variety of computer languages (many are already available)
• The core API logic resides on the API server, allowing consistent behavior across all client libraries
• New cloud providers can be added to the API without needing to change the clients

The driver for GoGrid is currently under development but will support the following actions:

• Create new instances
• Stop running instances
• Reboot running instances
• Destroy instances
• List all/get details about hardware profiles
• List all/get details about realms
• List all/get details about images
• List all/get details about instances

Deltacloud is hosted at Apache incubator which allows it to benefit from the Apache governance model. This means that discussion, changes and updates are based on individual contributions.

For more information on the Deltacloud initiative, please visit the project hosted on the Apache incubator site.

This month, we tried something a bit different with our Customer Newsletter for August 2010 in that we sent the newsletter from our Service Teams. In case you didn’t already know this, every GoGrid customer is assigned a service team to help our customers manage and grow their GoGrid account and answer any questions customers may have. The teams consist of an Account Manager, a Support Specialist, a Billing Specialist, a Data Center Technician and a Network Specialist.

Even though it is the summer here, we are hard at work growing the GoGrid product and feature offering as we strive to live up to our new Unique Value Proposition: “Complex Infrastructure Made Easy™”.

Below is the full content of the August 2010 Customer Update newsletter that went out to all GoGrid customers:

Hello,

It’s August and we’ve hit our stride. The release of GoGrid version 3.0 was a major milestone and we’ve continued at a steady pace for feature parity in our new US-East-1 data center. In this edition of the newsletter, you’ll discover how our new features and system enhancements can make an immediate impact on your GoGrid Cloud.

Specific topics include:

• New Features and System Enhancements
• Unique Value Proposition
• Partner News
• CDN News
• Events and Other News

New Features and System Enhancements

---

F5 Load Balancer Support US-East-1

In June, we opened our new US-East-1 data center in Ashburn, VA, which allows for immediate provisioning of virtual servers with GoGrid base images. We’re happy to announce that free F5 load balancing is now available for immediate deployment in the US-East-1 data center as well.

You can use the GoGrid portal or the API to provision virtual servers and F5 load balancers in real time  and in either of our locations: US-West-1 or US-East-1.

16 GB RAM Virtual Servers

In April we began the process of upgrading our node hardware with the latest Intel Nehalem chipset architecture allowing for 25-50% more CPU for virtual servers. We also gave Cloud Servers greater CPU allocations and now have included the ability to provision 16 GB RAM virtual servers in both GoGrid data centers.

As new node capacity continues to be built, we expect to soon have enough new Nehalem nodes to handle 100% of new virtual server adds.  Currently, most virtual servers will be added to new nodes, but depending on demand, some virtual servers may be deployed on non-Nehalem nodes and therefore will receive the older CPU allotments.

Below are some FAQs for this new feature:

Q: Are the new 16 GB virtual servers compatible with my existing MyGSIs and PGSIs?
A: Yes, all of your existing MyGSIs and PGSIs are compatible with the 16GB RAM virtual servers, and that option will automatically be made available with existing compatible MyGSIs and PGSIs. Note that only server images with a 64-bit OS will have the option to add a 16GB RAM server.

Q: Are there any restrictions with the types of operating systems I can use?
A: Yes, there are some restrictions with Windows operating systems due to Microsoft licensing constraints. Please see the chart below for specifics:

New Virtual Servers
RAM                Cores               Disk Storage
16 GB              16 (Linux)          960
8 (Windows)
Existing Virtual Servers
RAM              Cores               Disk Storage
16 GB            8                       960
If you have questions, please contact your dedicated GoGrid account team or GoGrid technical support.

Six New Images for GoGrid Dedicated Servers

We now support 32-bit and 64-bit versions of Ubuntu, CentOS, and Red Hat Enterprise Linux (RHEL) for our GoGrid Dedicated Servers. All of the existing images continue to be supported and are still available.
See below for new image details:

• Ubuntu 10.04 (32-bit) Dedicated Server
• Ubuntu 10.04 (64-bit) Dedicated Server
• CentOS 5.5 (32-bit) Dedicated Server
• CentOS 5.5 (64-bit) Dedicated Server
• RHEL 5.5 (32-bit) Dedicated Server
• RHEL 5.5 (64-bit) Dedicated Server

Make an Impact on Your GoGrid Cloud

At GoGrid you have a dedicated service team with cross-functional expertise that is available to help you every step of the way. Our team of experts can translate your business and technical requirements into the best solution for you.

Interested in or have questions about:

• Getting started with Cloud Storage
• Deploying Cloud Servers and F5 load balancers in the US-East-1 data center
• Hardware and software firewall solutions
• Leveraging F5 load balancing
• Provisioning a Partner GSI (PGSI)
• Building a Hybrid solution

Contact us directly to discuss getting started with a GoGrid solution. We will be happy to coordinate a technical and strategy call, as well as a consultative GoToMeeting on how to use our system.

Unique Value Proposition

---

Help Improve GoGrid and You Could Win an iPad

As part of our continued effort to provide a positive experience for our customers we are asking for your feedback. Below is a link to our quarterly customer survey. Simply fill out the survey and you’ll be entered to win an iPad. The survey is super quick (only a minute or two of your time) and will help us assess our ability to serve our customers and identify areas where we need to improve. This is your chance to provide input on the features that will help you. Here’s the link so that you can participate: http://www.surveymonkey.com/s/D7ZDKTY

Partner News

We continue to add exciting new partners to the GoGrid Exchange. In the past month we’ve added several new Partner GoGrid Server Images (PGSIs) to the GoGrid catalog. You can deploy an image directly from the GoGrid Portal.

NEW to the GoGrid Exchange:

• Trend Micro Deep Security – Trend Micro Deep Security 7 provides advanced protection for servers in the dynamic data center, whether physical, virtual, or in the cloud. Deep Security combines intrusion detection and prevention, firewall, integrity monitoring, and log inspection capabilities in a single, centrally-managed software agent.

• Automated Security Assessment Program (ASAP) – Cranium Solutions’ flagship product, ASAP™ (Automated Security Assessment Program), is the first fully automated end-to-end network security testing solution that runs as a loud-based service.  Businesses can perform simple and automated Vulnerability Assessments and Penetration Tests of any systems they have deployed on the GoGrid Cloud.

• enStratus – enStratus is a cloud infrastructure management platform for enterprise-class applications in private and public clouds including GoGrid. enStratus delivers governance for the cloud through a patent-pending security architecture and a range of management capabilities including financial controls, role-based security, auto-scaling, and auto-recovery.

UPDATES to the GoGrid Exchange:

• BitNami WordPress 3.0 – WordPress is a state-of-the-art publishing platform with a focus on aesthetics, web standards, and usability. WordPress is both free and priceless at the same time. The project was started in 2003. Since then it has grown to be the largest self-hosted blogging tool in the world, used on millions of sites and seen by tens of millions of people every day.

• BitNami SugarCRM 5.5.2-0 – SugarCRM is a flexible customer relationship management solution for companies of all sizes. SugarCRM can be easily customized and integrated with other software to allow companies to build and maintain a more flexible system. The core functionality includes sales force automation, marketing campaigns, support cases, project management, and calendaring.

• GigaSpaces XAP – GigaSpaces Technologies is a leading provider of a new generation of application platforms for Java and .Net environments that offer an alternative to traditional application-servers. The company’s eXtreme Application Platform (XAP) is a high-end application server designed to meet the most demanding business requirements in a cost-effective manner. It is the only product that provides a complete middleware solution on a single, scalable platform.

Webinar: Sentrigo and GoGrid

Title: Learn to Build a Stable, Scalable, and Secure Databases in the Cloud
Date and time: Thursday, August 12, 2010 10:00 am – 11:00am PDT
In this webinar, GoGrid and Sentrigo, a leader and innovator in database security, share our expertise on database deployments and real-time security in this latest frontier of information technology. Click here to register, or paste http://go.gogrid.com/e/3442/7YXX2/RXDE/7101902 into your browser.

CDN News

---

18 Global PoPs

GoGrid is pleased to announce that our Parisian CDN PoP (Point of Presence) is now fully functional, bringing the total number of global GoGrid CDN PoPs to 18. Also new to the GoGrid CDN are upgrades to our Flash Live Streaming Platform and new HTTP Rules Engine.

There are no fees to have a CDN account provisioned. Learn more or have yours created today.

Flash Live Streaming Platform

If you’re looking to stream live Flash video, then our new offering Live StreamCast offers superior scalability and performance over our previous Live Streaming service.
Enhancements include:

• Automatic redundancy and failover
• A new live publishing point in Australia
• Simpler URL structure
• Improved scalability

We’ll continue to offer and support our existing Flash platforms: Flash On-Demand and Flash via HTTP progressive download.

HTTP Rules Engine

The HTTP Rules Engine gives you highly granular control over how, when, and if your content is served. The possibilities are endless, but the end result is much more control over the security, cost, and burden of your content delivery.
The HTTP Rules Engine allows you to:

• Easily block content leeching and hotlinking
• Slash bandwidth costs by specifying precise “origin pull” update intervals for each URL or object
• Reduce load on your origin servers by offloading your existing rules to our processing engine

The cost is $299.99/month and $299.99 for setup.
For more information, consult your Service Team Account Manager.

Events and Other News

---

Net Promoter Survey

If you’ve filled out our Net Promoter Survey, Thank You! We recently sent you a short survey asking you if you would recommend GoGrid to a friend or colleague. Your feedback is very important to us and will be reviewed closely by our executive team. If you haven’t had a chance to fill out the survey, there’s still time.

GoGrid 3.0 Webinar

On June 28, we hosted a webinar featuring the release of GoGrid version 3.0 and our new East Coast data center. If you were unable to attend or would like to review the recording, visit our blog.

As always, you can find more information about GoGrid by visiting the following links:

• GoGrid Product Site – www.gogrid.com
• GoGrid Blog – blog.gogrid.com
• GoGrid Wiki – wiki.gogrid.com
• GoGrid Forums – forums.gogrid.com

If you have any questions please feel free to contact your service team directly.

Thank you,

[Service Team Account Executive]

Your Service Team

Don’t forget to follow GoGrid on Twitter (@GoGrid) as well as become a fan (or like us) on Facebook. If you have any questions or suggestions, feel free to leave a comment on this post or contact your Service Team.