Copyright 2002-2008 Data Mobility Group, LLC. All Rights Reserved.
Reproduction of this publication
without prior written permission is forbidden. Data Mobility Group believes the statements contained herein are based on accurate and reliable information. However, because information is provided to Data Mobility Group from various sources, we cannot warrant that this publication is complete and error-free. Data Mobility Group disclaims all implied warranties, including warranties of merchantability or ftness for a particular pur- pose. Data Mobility Group shall have no liability for any direct, incidental, special, or consequential damages or lost profts. The opinions expressed herein are subject to change without notice. datamobilitygroup.com Long-Term Road Test Data Mobility Group 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Joseph Martins February 6, 2008 jmartins@datamobilitygroup.com High Value Remote Offce Data Protection with EMC Avamar This Data Mobility Group report offers a brief overview of remote offce 1 data protection challenges and opportunities for 2008 along with an objective assessment of one solution, EMC Avamar, based on more than twelve months of frst-hand experience with the product. In discussing the results of DMGs evaluation, this report will provide insight into the benefts and appropriate application of EMC Avamar technology in remote offce environments. Some technologists continue to perceive data protection solely as a way to mitigate, minimize, and avoid riska glass half empty. The savviest managers also view it as an opportunity to improve operational effciencies. They know the right combination of data protection technologies can optimize capital and operational expenditures, minimize the impact of unpredictable events, and reduce the risk of regulatory and legal non-compliance. They also understand that a nonexistent, poorly executed, or outmoded data protection program is the equivalent of throwing money and caution to the wind. In their quest to protect the crown jewels, information technology (IT) managers face considerable pressure from senior management to provide a strong business case for data protection investments in terms of business continuity and risk management. 2 While an increasing number of IT managers understand the benefts of high value data protection and make their cases to senior management, their departments barely have the resources to keep up with rapid data center growth, let alone to 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com bring the ineffcient infrastructures of scores of remote offces into the 21 st century. As a result, many remote offces continue to lack adequate disaster recovery and operational data protection 3 , and more than half of them are still completely reliant on high-touch, hit-or-miss tape backups. This is especially troublesome given the large percentage of corporate information assets residing outside the data center, in remote offces, off the IT departments radar and beyond its control. 4
A commitment to high value data protectionfrom the data center out to the edgerequires a willingness to let go of the past and embrace change, but the long-term cumulative payoff is worth it. Take the slow-moving healthcare industry as one example in which high value data protection has the potential to deliver considerable savings. In 2002, The United States Census Bureau counted over 704,000 healthcare and social assistance establishments in the US, that employ more than 15 million people. 5 The following table includes data as reported by the top 50 frms in each sub-sector: NAICS Subsector # locations # employees Payroll ($1000) 621 Ambulatory Health Services 10,640 416,606 17,846,997 622 Hospitals 1,356 1,350,160 51,132,875 623 Nursing & Resid. Care 7,770 559,422 12,025,148 624 Social Assistance 5,285 151,741 2,704,860 Table 1: Top 50 Firms In Each Healthcare Sub-sector (U.S. Census Bureau) 6 Two hundred frms account for an astonishing 25,051 offces. They employ nearly 2.5 million people and manage sensitive HIPAA 7 -regulated information about tens of millions of patients. Even minuscule improvements in operational effciency have a substantial impact on the combined $84 billion dollar payroll. Rough calculations reveal that saving employees just one minute in their daily routines would reclaim $13,415 dollars per daymore than $3.35 million dollars per company, per year. 8
Organizations can choose to maintain the status quo until, in the not-so-distant future, unrelenting data growth and manageability problems cause their existing infrastructures to collapse under pressure. Or, they can choose to tackle the problem sooner rather than later, optimize expenditures, minimize the impact of unpredictable events and reduce exposure to non-compliance liabilities by applying some of the lessons learned in their data centers to remote offces at the edge of their networks. 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com Challenges and Opportunities of Remote Offce Data Protection From a business perspective, data centers and remote offces share many of the same underlying information management constraints. That is to say, their digital asset types are the same, their legal and regulatory obligations are the same, and their employees (and customers) typically require similar levels of service, continuity, and security. While there are always exceptions to the rule, this degree of commonality usually enables organizations to standardize everything from the way data is classifed and workfows defned, to the manner in which digital assets are protected and preserved. The similarities end there. Data centers are better equipped and better staffed with more advanced technology and on-site technical experts. In contrast, remote offces too often rely upon outmoded technologies and processes, managed by less profcient or offsite personnel. In the following table, DMG offers a summary of several remote offce challenges along with corresponding opportunities for high value data protection: Table 2: Remote Offce Backup Challenges Challenge Opportunity Solution Limited or nonexistent onsite technical resources to manage and troubleshoot data protection operations and media Eliminate the need for onsite IT expertise for operational backup and recovery. Minimize disruptions to employee productivity. Consistent, centralized backup management and monitoring. Self- service local recovery. Disaster recovery from offsite tape, disk or 3 rd party managed services. Limited or nonexistent onsite compliance experts. 9 Reduce the cost of compliance and risk of non-compliance at remote sites. Consistent, centralized policy management and monitoring. Limited, non-existent, inconsistent or undocumented data retention and protection processes. Same as above. Same as above. Inconsistent network bandwidth from one facility to the next. Lower network bandwidth consumption and costs by minimizing backup traffc on the wire. Leverage backup data reduction technologies (e.g. de-duplication), which lower required network bandwidth Data growth makes it increasingly diffcult and expensive to back up via WAN Same as above. Same as above. 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com Challenge Opportunity Solution Inconsistent software and hardware confgurations from one facility to the next. Lower long-term IT overhead and capital expenditures. Consolidate and standardize as existing technology is end-of-lifed. Ineffciencies in network bandwidth consumption, storage consumption, backup and recovery times, diminished service levels, lost productivity and opportunity. Reduce network bandwidth and storage consumption. Eliminate the need for lengthy backup or recovery windows that negatively affect operational effciency. Leverage backup data de- duplication technologies, which lower required network bandwidth and disk storage. Offer fast, granular operational recovery via a non-technical self-service user interface. Limited resources already stretched to the limit. Protect more data, for longer periods of time, at a lower cost with fewer IT resources. Leverage technologies that minimize capital and operational expenditures The Accelerative Impact of Data Growth Organizations ignore data growth at their own peril. Near exponential data growth sneaks up on the unprepared, and DMG predicts the current situation is only going to accelerate over the next several yearsespecially the growth of multimedia and messaging content. Today some offces manage a handful of gigabytes or terabytes, but before long they will fnd themselves managing dozens or hundreds of terabytes. Worse, their existing data protection schemes built on a virtual mountain of redundant data (i.e. snaps, clones, mirrors, etc.) will continue to multiply the effects of that growth. The impact of data growth fguratively hit home when massive one terabyte (TB) disk drives shipped for the frst time in early 2007. 10 To put that news into perspective: one of the largest not-for-proft hospital networks in the United States, consisting of 15 hospitals, consumed just 1.5 TBs of storage in 2001. By early 2005, the networks storage consumption had increased to 138 terabytes. Today, consumers can purchase terabyte drives to store their vacation footage, photos and personal information at home. Savvy consumers who understand the value of data protection have already begun leveraging both local external disk drives and online managed data protection services, such as EMC Mozy, to protect their personal information assets. Consumer awareness of data protection at home has begun to drive demand for similar user-friendly, self-service, rapid recovery data protection at the offce. 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com Living with EMC Avamar December 6 th, 2007 marked the one-year anniversary of Data Mobility Groups second EMC Avamar evaluation. During the frst evaluation, in 2004, as part of our CODiE Award judging responsibilities, DMG spent two months testing the products advanced data protection features against Avamars claims. 11 Unlike traditional backup and recovery software, EMC Avamar utilizes patented global de-duplication technology to reduce the size of backup data at the sourcebefore it is transferred across the network and stored to disk. Avamar excelled in several areas including backup and recovery performance, and disk storage optimization. DMGs only advice to Avamar at the time: streamline the user interface a bit more and decouple the software from the hardware for customers who plan to use their own equipment. Fortunately, the decoupling, along with development of an enhanced user interface and a host of other improvements, was already underway. With a long list of new enhancements under its belt, Avamar returned to DMG for a second evaluation toward the end of 2006. This time around DMG chose to deploy Avamar for a year-long test drive in its own offce beginning with approximately 200 GBs of primary datato directly observe the long-term impact of Avamars de-duplication technology, as well as the ways in which non-technical users might leverage the system beyond data protection. 12
After the completion of DMGs frst evaluation, there was no question about the validity of Avamars core value propositions. The products ability to deliver fast, daily full backups across existing networks, reduce the required backend disk storage, conserve network bandwidth, and reduce recovery times was apparent from just a handful of tests using a mixed data set of Microsoft Offce fles, multimedia content, email and database fles. 13 Our results simply confrmed the experiences described by EMC Avamars existing customers in numerous interviews and articles. Still, we were determined to experience and measure the extent of EMC Avamars value for ourselvesover a much longer period of timein a small offce environment with limited on-site technical resources, ordinary network bandwidth,
and a strong operational dependency on information assets. 14
The best offce products hide their complexity beneath a veneer that makes them accessible to even the most technically challenged among us. Quite frankly, if we cannot install and support a product here in our offce, then it does not belong in any remote offce. EMC Avamar passed with fying colors. 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com Selecting and Setting Up EMC Avamar In DMGs opinion, prepackaged solutions are perfect for remote offces, especially those with limited IT resources and a need to get up and running as quickly as possible. Avamars fexible deployment options include the EMC Avamar Data Store, which provides pre-packaged Avamar software with EMC certifed hardware to simplify ordering, installation, and service. It is available in a variety of confgurations to meet the needs of small offces to datacenters. Customers can also choose to purchase EMC Avamar software separately and install it on a range of industry standard EMC certifed servers. And the Avamar Virtual Edition is, we believe, the industrys frst virtual appliance for backup and recovery, leveraging an existing VMware ESX server and storage. As of the time of publication, EMC Avamar had more than 1,000 customer deployments. The product is typically deployed to back up data residing in remote offces, VMware environments, and data center LANs enabling daily full backups for just a few to many thousands of clientswith network bandwidth ranging from snails pace 56K to multiple OC-12 links. We were told that the largest Avamar deployment to-date is at a global IT networking manufacturer, which uses Avamar to protect 235 TB of primary data, spanning 14 US sites and 12 international sites, with replication to 6 hub sites. Our plans were not quite so ambitious. We utilized an EMC certifed industry-standard 2U Dell PowerEdge server equipped with 1.8 TB in a 6 x 300 GB SCSI array. We completed the installation ourselves, with minimal guidance from an EMC engineer via phone and a copy of the manual, in less than 30 minutes. 15 In under an hour we had installed the necessary client agents and confgured our frst backups. While we are generally not fond of agent-based architectures, Avamars proved to be lightweight and low maintenance. Avamars client-side agents and application plug-ins perform de-duplication at the source (i.e. on the clients), before backup data is transferred across the network and stored to disk. Avamars patented de- duplication technology chunks backup data into sub-fle variable length segments to ensure that only new, unique segments are moved and stored across sites and servers. As a result, Avamar provides daily full backups across existing LAN/WAN links and data can be encrypted for security, eliminating reliance on tape and the risk of shipping tapes offsite. These are just some of the capabilities that make Avamar an excellent choice for companies with a large number of offces and clients distributed across a WAN. DMG strongly suggests that customers plan ahead. Knowing in advance which clients, applications, and fles must be protected will expedite the installation and confguration process. The same applies to backup schedules and any other backup solutions with which Avamar must cooperate. EMCs professional services are available to help customers size their environments, select the appropriate confgurations and install them. The following 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com table shows our confguration as of December 6, 2007, which had changed somewhat since we began the evaluation in 2006: Clients Server Type O/S as of 12/6/2007 Data File server 1 AMD Athlon 1.4GHz, 768MB RAM (custom built) WinXP SP2 36.8 GB Media Server 1 Intel Core 2 Duo, 2.2GHz, 1GB RAM (custom built) WinXP SP2 65 GB Media Server 2 Intel Core 2 Duo 2GHz, 2GB RAM (Macbook) Mac OSX 10.5 71 GB File server 2 Intel Core 2 Duo 2GHz, 2GB RAM (Macbook) Mac OSX 10.4.5 19 GB Database Intel Core 2 Duo 2.2GHz, 1GB RAM (HP DV9000) WinXP SP2 2 GB Lifesaver Single Node EMC Avamar Server Dell PE 2850 , 1.8TB RHEL Linux - TOTAL 193.8GB Table 3: DMG Offce Hardware Confgurations General Impressions Ease of Use Protecting DMGs information assets had never been easier. Like many small offces, we relied upon a mixed bag of data protection schemes from DVD-R and external hard drives to online managed services. Given the opportunity, later this year we hope to test EMC MozyEnterprise data protection service as yet another possible alternative to local disk-to-disk (D2D) backup with offsite tape. Since installing the system in 2006, we have contacted EMC support only two times for assistance: once to upgrade Avamars software to a new release, and once to request the Mac OS X agent released midway through 2007. That is it. We were very impressed with the responsiveness and the follow-through of the EMC customer support team. And while the upgrade process is still manual, it is simple enough so as not to be disruptive. We would like to see Avamar automate the process at some point. Nevertheless, Avamars system is about as low- touch as it gets in the world of data protection. We were not able to test Avamars ability to centrally manage multiple clusters in our single node environment, but we have read reports that it works quite well. Certainly, the user interface is one of the best we have seen (and we have seen many) and it really does take just a few clicks to perform a variety of functions, including backup, recovery, and at-a-glance reporting. 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com EMC Avamar (daily fulls) Traditional backup methods (weekly fulls and daily incrementals) Traditional backup methods (daily fulls) Total Backup Storage Consumed Over 13 Months ~18 TB ~76 TB < 1 TB For 13-months we performed daily full backups with EMC Avamar and used less than 1 TB of total disk storage. Traditional backup methods would have required over 76 times more disk storage to perform the same number of daily full backups. (mix of database and MS Offce fle data) Optimized Disk Storage Avamars de-duplication technology enables daily full backups across existing networks and also signifcantly reduces the required total backend disk storage. During our initial 2006 testing, we dramatically reduced our required disk capacity (reducing the size of backup data by more than 250x on at least one server), and Avamars effciency only increased over the twelve-plus months that followed. As the amount of protected data increased, the likelihood that new data shared common sub-fle segments with existing data increased as well. 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com Gigabytes of data that would have otherwise been redundantly backed up over and over were whittled down to the bare minimumoftentimes just a few megabytes each day. Typical sub-fle commonality was greater than 99% on the fle serversmeaning Avamar protected all of the data, but only moved the new, unique sub-fle data (less than 1% per day) during daily full backups. 16 17 Bandwidth Effciency DMG does not have a WAN, nor do we have an ultra-high-speed connection to the internet. Fortunately, with EMC Avamar, there is really no need for it. After the initial backup, our LAN and low-speed satellite internet connection were more than adequate to handle the backups. Avamars global de-duplication at the source, combined with its compression prior to transport, minimizes the amount of data moved from protected clients to the Avamar server, or from Avamar server to another for disaster recovery. The impact was clearly visible during consecutive backups when backup windows shrunk to a fraction of their previous length. Imagine having the ability to protect a couple hundred gigabytes of datafull daily backups, not incrementalsacross several clients, in just a few minutes. Dual Purpose Backup and Archive Avamars astounding space savings and bandwidth effciency made it feasible and simple for DMG to do full daily backups for the entire year. The thought of 52 weekly full tape backups, with or without daily incrementals, is both a fantasy and nightmare for any IT manager. Scaled up to a typical remote offce environment, it would be cost-prohibitive using most traditional backup systems, and virtually impossible to manage. Our total disk space consumption as of December 2007 was still well under our Avamars capacity. The ability to store so much protected data for so long at such a competitive price point enables Avamar to fulfll a secondary rolethat of a long-term archiveat no additional cost. We previously believed that a combination archive and backup was neither feasible, nor desirable, but we are now convinced it is not only feasible, but highly effective. Users who often fnd themselves searching for material created months earlier can, as we ourselves have done on multiple occasions, quickly browse the archive to fnd and copy what they need. Try doing that with tape or other removable media. Fast, Single-Step Recovery Earlier in this report we described the dramatic cost savings made possible by saving employees just one minute of time each day. For more than a year now, EMC Avamar has saved DMG time every day by enabling us to 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com quickly recover data via Avamars virtual fle system as easily as we might search our desktops using Windows Explorer or Apples Spotlight. No fddling with DVDs to fnd the fles we need or retrieving tapes from offsite storage vaults. No need to restore an entire volume to a server just to recover a single fle or folder. And most importantly, just a couple clicks from search to restore. Reliability and Security We have read about Avamars patented redundant array of independent nodes (RAIN) technology, which enables Avamar to survive a node (server) failure without interruption, but we have not yet had a server failure (or even a disk drive failure), so there is little we can claim about it at this point. As we have time in 2008, we hope to dig a little deeper and amend this report accordingly. In regards to security, Avamar natively encrypts data with its own protocol, and it supports AES 128 bit encryption as well. IT can also restrict user access to fles with Avamar's built in security tools. As a result, companies can safely use existing WAN bandwidth for backup, recovery, replication, and disaster recovery. Avamar essentially eliminates the need to ship tapes offsite, and the risk of losing unencrypted data. Given recent high-profle stories of backup data theft and lossespecially incidents that involved tapeIT managers will welcome the extra protection. In a NutShell In March 2005 DMG published a very well-received and widely-distributed report titled Is Tape Really Cheaper Than Disk? Three years later, our conclusion that tape was cheaper than disk no longer accurately refects the data protection marketplaceat least not in the context of remote offces. DMG believes EMC should do something about Avamars pricing structure to make it more accessible to smaller shops, but it is still very competitive. IT managers can compare the loaded cost per gigabyte of tape versus disk, but the high value data protection provided by a product such as EMC Avamar is indisputable and transcends ordinary total cost of ownership (TCO) calculations. The systems ability to conserve network bandwidth, and minimize disk storage consumption has to be experienced to be believed. Even at a rate of 20-50x we would have been pleasedat 250x we were absolutely ecstatic. Overall, DMG has been very pleased with the product and EMCs responsive technical support team. Most impressive, however, was the additional business value DMG experienced throughout 2007beyond its original expectations. 2002-2008 Data Mobility Group. All Rights Reserved. 76 Northeastern Blvd. Suite 29A, Nashua NH 03062 Phone: 603.835.6141 Fax: 877.254.4886 Data Mobility Group www.datamobilitygroup.com DMG is aware that Avamar is not the only solution available for remote offce data protection, but we feel it is one of the best, and we have more than 12 months of in-house experience to back up that claim. If EMC Avamar isnt on your short list of data protection alternatives for your remote offce, it should be. Questions about our experience with EMC Avamar? Drop us a line. Endnotes 1 DMGs use of the terms satellite offce and remote offce throughout this report should be interpreted to mean offces with roughly 10-250 people, functioning as remote facilities for larger organizations. 2 In recent years, traditional total cost of ownership (TCO) and return on investment (ROI) analyses have been supplemented with net present value (NPV) appraisals. 3 Estimates obtained from a number of publicly available sources online vary, but most exceed 55%. 4 Estimates obtained from a number of publicly available sources online vary from as low as 30% to greater than 60%. 5 An establishment is a single physical location at which business is conducted and/or services are provided. It is not necessarily identical to a company or enterprise, which may consist of one establishment or more. 6 U.S. Census Bureau, 2002 Economic Census, Healthcare and Social Assistance, Geographic Area Series (August 2005). This report supersedes any previous industry series reports. 7 Health Insurance Portability and Accountability Act 8 The value of one minute of productivity gained or lost per day, per company, summed over a period of 250 work days. 9 The Bank Secrecy Act, Basel II, Californias Senate Bill 1386, the European Union Data Protection Directive (not including the individual data protection and privacy policies of its 27 member and 3 candidate countries), the Federal Information Security Management Act, Federal Rules of Civil Procedure (FRCP), Gramm-Leach-Bliley Act, Health Insurance Portability and Accountability Act (HIPAA), Payment Card Industry Data Security Standard, Personal Information Protection and Electronic Documents Act (PIPEDA), Sarbanes-Oxley (SOX), and the USA PATRIOT Act just to name a few. 10 The one-terabyte Hitachi Deskstar 7K1000 began shipping in March 2007 11 The frst evaluation took place prior to EMCs acquisition of Avamar when the product was known as Avamar Axion. 12 Frankly, there is no better way to assess the long-term impact and value of technology than to live with it 24-7 for an extended period of time. 13 A blend of Microsoft Offce documents, Adobe fles (PDF, InDesign, Photoshop, Illustrator, etc), Internet-related fles (HTML, CSS, XML, PHP, etc), CAD drawings, several 3D modeling, video and audio formats, Microsoft Exchange, a variety of source code fles, compressed archives, and MySQL databases. 14 Although DMG analysts have technical backgrounds, our aim was to play the part of relatively non-technical end users. Connectivity provided by Dish Network satellite broadband advertised as 1.5 Mbps/256 kbps, but the actual download average is less than half that amount. 15 The documentation is excellent and thorough. 16 Even DMG projects that involved extensive database usage or a sharp increase in multimedia content had only a modest impact on the overall growth rate. 17 Values were calculated from backup log data and rounded to the nearest tenth.