Posts Tagged ‘backups’

 

How To Optimize Your Database Backups and Text File Compression with pbzip2 and pigz

Thursday, February 9th, 2012 by

Recently, GoGrid was examining performance enhancements on several internal processes; among these enhancements was switching from standard gzip to “pigz”. Since I had never heard of this “pigz”, I was intrigued by this supposed “parallel” implementation of gzip; meaning it uses all available CPU’s/cores unlike gzip. This prompted me to ask, “I wonder if there is a parallel implementation of bzip2 as well”, and there began my endeavor.

pigz and pbzip2 are multi-threaded (SMP) implementations of their respective idol file compressors. They are both actively maintained and are fully compatible with all current bzip2 and gzip archives.

If you’re like me, you might’ve stayed away from using gzip or bzip2 due to the single-threaded aspect. If I try to compress a, let’s say, 2GB file, the system becomes rather sluggish; the reason being is that the “compression tool of choice” uses almost all of 1 core of today’s multi-core, multi-CPU systems and creates an uneven load between the cores, causing the CPU to operate very inefficiently.

In this example I have a .tar file with several databases in it, which totals 1.3GB. The system in question is a GoGrid dedicated server with 8 cores. The server’s load is around 1 and is a production database server.

Using bzip2, the file took approximately 6 minutes and 30 seconds to compress. Yikes!

bzip2

(more…) «How To Optimize Your Database Backups and Text File Compression with pbzip2 and pigz»

The Importance of Building Your Cloud Infrastructure the RIGHT Way

Thursday, July 21st, 2011 by

The cloud is great for so many things. You can create a web presence in a matter of hours or completely implement an N-tiered, redundant, elastic, secure globally-available cloud topology. Spinning up infrastructure via a web portal or API in minutes via a few clicks of a mouse is a dramatic transformation from the days of racking and stacking servers, untangling miles of cat5/6 cables, connecting load balancers and firewalls to the mix and hooking up storage devices. And let’s not forget about physical security, power supplies, cooling and network redundancy. The neat thing about the cloud is that all of the stuff has become really easy to do and you can do it very quickly.

GoGrid has a long history of enabling IT infrastructure solutions for companies across the world. We have built out core services and offerings to allow businesses to build want they want quickly, efficiently and with state-of-the-art cloud technology. But just because you have great tools at your disposal doesn’t mean that your cloud environment will magically create itself. And that is something that we realize and understand at GoGrid.

clip_image002

Architect for Success

Cloud computing can be almost magical at times, but we need to remember the processes and best practices for security and ensuring redundancy that we are accustomed to using, and adapt and use them within the cloud as well.

A few weeks ago, I wrote a post “Things to Think About When Building Secure Infrastructure” where I made a few points about “assumption,” namely, assuming that whatever cloud vendor you choose, they will take care of everything for you. Regardless of the cloud vendor, you need to do your due diligence and update your standard operating procedures to reflect how cloud computing works. It is different than traditional IT in many ways. For example, in the GoGrid cloud, you can create a cloud server, harden it with security software and configurations and then save it as a MyGSI (as “server image”). Then, as you need to scale out your infrastructure, you can do this not only quickly, but securely as well, by deploying clones or instances of that hardened server. With a traditional, physical deployment, it takes much longer and there is no guarantee that you will have each and every security patch in place on every server.

(more…) «The Importance of Building Your Cloud Infrastructure the RIGHT Way»

“In Cloud We Trust?” ReadWriteWeb Asks & My 2 Cents

Monday, January 26th, 2009 by

readwriteweb_logo Today, Sarah Perez of ReadWriteWeb posted the question, “Do you trust the cloud?” to FriendFeed and wrote about her findings in the article “In Cloud We Trust?” The problem is, I believe the question itself was too vague. But this “finger to the air to test the wind direction” did spark quite a bit of discussion and further made me realize that the public in general doesn’t fully yet understand the full spectrum of Cloud Computing (and this was even within a social media/tech-savvy audience).

As is evident from the 90+ comments that popped up within 18 hours of posting the question, people have a lot to say about the subject. The important thing to consider here is the lack of granularity of the question and the range of responses. To really ask and analyze the question better, one must fine-tune it more to the detailed components of what makes up Cloud Computing, namely: Cloud Applications, Cloud Platforms, Cloud Aggregators, Cloud Extenders and Cloud Infrastructure. My guess is, most people responding to the question don’t truly understand the differences between these layers in the Cloud. Perhaps better, more focused questions would have been:

  • “Do you trust Cloud Applications like Flickr, Facebook and Gmail?”
  • “Do you trust Google or others with your critical data?”
  • “Do you see yourself using the Cloud as your primary or ancillary IT strategy?”

New_Cloud_Pyramid

It seemed to me that the common thread within the FriendFeed responses was that of FUD: Fear, Uncertainty, and Doubt. The question itself is phrased with an inherent FUD factor which can quickly skew the resulting answers. However, I do think that this question is important from a consumer standpoint, that of the everyday user of Gmail or Evernote or DropBox, for example. The biggest commonality that I saw from reading through all of the comments was that of “backups.” My read is that people are concerned that their data will be lost in some way, either by a company pulling the plug or a hard-drive crashing or just not being able to physically “touch” it.

From a consumer standpoint, this article is appropriate. As the amount of data that consumers produce in the form of emails or photos for example, continues to grow almost exponentially, they are realizing that storing this un-replaceable data in a single location is risky. Many back up this priceless data on external hard-drives or CDs/DVDs. Some seem to be venturing to the “Cloud” as a secondary redundancy, by using Cloud Storage to solve this.

(more…) «“In Cloud We Trust?” ReadWriteWeb Asks & My 2 Cents»