Corporate Backup and Redundancy

SmartKit

SmartKit

SmartKit Rep
Joined
Jun 29, 2008
Messages
8,218
Reaction score
856
Location
The Reef
Hi all

Recently a company, that I am not affiliated with, suffered a catastrophic RAID failure, first one of the drives and then the entire controller corrupted the entire array. As such recovery was done from a full backup and 3 weeks worth of incremental backups. This process took 4 days.

This is, of course, unacceptable and I was hoping someone could point me in the right direction as to whom to approach that might provide a better solution than the one currently implemented. The company is not enamoured with the current provider and I suggested approaching a few others companies to get ideas and possibly contract for future work.

The biggest concerns are the 1.2TB datastore and the 2TB Exchange Server.

In the event of a worst case failure they would like to resume within minutes, this would indicate the need for a mirror. Full restores and backups should be in the region of hours, not days. A proper offsite system for backups needs to be investigated.

If you can't recommend yourself, are there any other companies that you would? Keep in mind this is a large commercial enterprise that stands to lose millions in the event of a massive failure (fires, earthquakes, revolutionaries and alien attack are all possibilities ;)).

Thanks!
 
Is everything virtualized or not?

On the mirroring side one option I've recently been working with is Proxmox VE, a great virtualization tool that has integrated backup tools and supports high availability functions. From the sounds of it the company has sufficient resources to deploy redundant hardware so I would definitely look at setting up 3 hardware nodes configured with high availability. This way if one node goes down you shouldn't have any downtime.

Check out Proxmox (www.proxmox.com) as a possible solution. Here is a wiki article on setting up 2 node cluster with DRBD storage in HA. http://pve.proxmox.com/wiki/Two-Node_High_Availability_Cluster

You can still perform onsite backups as well as using an online backup service, you can never have too much redundancy.
 
Last edited:
A SAN, or Storage Area Network, is most commonly used in conjunction with incremental backup or mirroring to a remote location.

High Availability Clustering has no singular point of failure component, except if mechanisms employed within the HA setup fails which is really quite rare.
 
Which version of Exchange are they running?

1.2TB datastore? I'm guessing file shares etc?

nand said:
A SAN, or Storage Area Network, is most commonly used in conjunction with incremental backup or mirroring to a remote location.

High Availability Clustering has no singular point of failure component, except if mechanisms employed within the HA setup fails which is really quite rare.
Click to expand...

A SAN is not a HA device, I've seen many of them fail over the years, had two fail at one client last year. If you want to use a SAN in a HA environment you need two of them in a mirror.


Edit: OP are the servers physical or VM's?
 
Last edited:
Virtual Servers,Dual SAN datastores mirrored or block-level replication of VMs

I've tested Shadowprotect and Appasure
 
For Exchange, use DAGs on different servers for resilience (and potentially load balancers for your front-end servers; Zen have reasonably decent cheap load balancers), and some for of backup software for recovery.

For the datastore, it depends on what the datastore actually consists of. For a file server, you could use something like DFS.
 
In 2008 I moved my old Exchange 2003 to Google Apps. Now I can burn my servers and Everything is still running.

Once somebody asked me "what if Google goes down? What is your Backup Plan for that?" My answer was "If Google goes down... You better get yourself and your family to a safe place as doomsday is about to knock on our doors and email will not be important anymore".

Since 2008, only one downtime from Google -it was all over the news-.
 
webtailor said:
In 2008 I moved my old Exchange 2003 to Google Apps. Now I can burn my servers and Everything is still running.

Once somebody asked me "what if Google goes down? What is your Backup Plan for that?" My answer was "If Google goes down... You better get yourself and your family to a safe place as doomsday is about to knock on our doors and email will not be important anymore".

Since 2008, only one downtime from Google -it was all over the news-.
Click to expand...

Google Apps isn't really suitable for a large commercial enterprise. They tend to like keeping an eye on their information.
 
webtailor said:
In 2008 I moved my old Exchange 2003 to Google Apps. Now I can burn my servers and Everything is still running.

Once somebody asked me "what if Google goes down? What is your Backup Plan for that?" My answer was "If Google goes down... You better get yourself and your family to a safe place as doomsday is about to knock on our doors and email will not be important anymore".

Since 2008, only one downtime from Google -it was all over the news-.
Click to expand...

^^^ This. There is almost no reason to run mail infrastructure internally. Considering the TCO of exchange server (staffing, licensing, CALs, annual support license) you will find that Google Apps will be at least 40-50% cheaper compared to an on-premise Exchange installation. It is surprising that your company does not have either an active-active or at least an active-passive configuration.

For any compliance requirements you would use Google Apps Vault (http://www.google.com/enterprise/apps/business/products.html#vault).
 
Last edited:
Thanks everyone.

Just to answer a few q's:

They're running VMs, each running Windows, except for Exchange, which is on a separate server though could easily be imaged.

Current suggestions for backup and recovery include CA ARCserve® D2D, Symantec System Recovery, Dell AppAssure and Veeam. The mirroring software would be something new.

An online option is not a solution as the data is highly sensitive and several eyes need to be kept on it from a legal point of view.

So a lot of these options are great and I will make mention of them; however, I'm by no means and expert on networking and recovery and would need to present that opinion from a professional company.
 
phaktza said:
Thanks everyone.

Just to answer a few q's:

They're running VMs, each running Windows, except for Exchange, which is on a separate server though could easily be imaged.

Current suggestions for backup and recovery include CA ARCserve® D2D, Symantec System Recovery, Dell AppAssure and Veeam. The mirroring software would be something new.

An online option is not a solution as the data is highly sensitive and several eyes need to be kept on it from a legal point of view.

So a lot of these options are great and I will make mention of them; however, I'm by no means and expert on networking and recovery and would need to present that opinion from a professional company.
Click to expand...
Do yourself a favour and look at Acronis - either for Hyper-V or VMware. Unlimited VMs per physical host, plus support for Exch, SQL, AD and SP. Image snapshot technology, granular app restore (down to single mail, database, SP item). Not the cheapest, but we wouldn't be without it.
 
phaktza said:
Thanks everyone.

Just to answer a few q's:

They're running VMs, each running Windows, except for Exchange, which is on a separate server though could easily be imaged.

Current suggestions for backup and recovery include CA ARCserve® D2D, Symantec System Recovery, Dell AppAssure and Veeam. The mirroring software would be something new.

An online option is not a solution as the data is highly sensitive and several eyes need to be kept on it from a legal point of view.

So a lot of these options are great and I will make mention of them; however, I'm by no means and expert on networking and recovery and would need to present that opinion from a professional company.
Click to expand...

Do they want a HA solution or just backups that they can restore faster?

The VM's are they on Hyper-V or VMware and which version?
 
grim said:
A SAN is not a HA device, I've seen many of them fail over the years, had two fail at one client last year. If you want to use a SAN in a HA environment you need two of them in a mirror.
Click to expand...

While I agree that a SAN is not a HA device, a proper SAN doesnt just fail. They have full redundancy built in from drives through boards and power supplies. A NAS is not the same as a SAN.
 
Rocket-Boy said:
While I agree that a SAN is not a HA device, a proper SAN doesnt just fail. They have full redundancy built in from drives through boards and power supplies. A NAS is not the same as a SAN.
Click to expand...

You'd be surprised, the two that failed last year were a HP MSA2012FC and HP P2000 G3, both connected via two fabric switches to multiple HBA's in the Hyper-V hosts running in a cluster

The MSA2012 lost both it's storage controllers at the same time somehow and the P2000 lost a volume on the RAID

The setup looks something like this:SAN.jpg
 
grim said:
You'd be surprised, the two that failed last year were a HP MSA2012FC and HP P2000 G3, both connected via two fabric switches to multiple HBA's in the Hyper-V hosts running in a cluster

The MSA2012 lost both it's storage controllers at the same time somehow and the P2000 lost a volume on the RAID

The setup looks something like this:View attachment 106346
Click to expand...

That is pretty weird because that setup looks effective.
Waaay too often I hear people calling a NAS a SAN so was just checking :p
 
Rocket-Boy said:
That is pretty weird because that setup looks effective.
Waaay too often I hear people calling a NAS a SAN so was just checking :p
Click to expand...

No worries, also hear it quite often.

I still don't trust a single SAN no matter how many redundancy features it has plus they aren't the greatest for massive workloads.

A very popular email continuity and archiving provider in SA don't use a single SAN because of throughput and cost issues, it's cheaper for them to just run servers with local storage and faster because of the sheer volume of servers they have. They had just over 11PB of storage in SA last time I spoke to one of their techies :eek:
 
You must log in or register to reply here.
Back
Top
Sign up to the MyBroadband newsletter
X