Advanced remote data management and movement technology, such as that incorporated into Adaptec Snap EDR now makes it possible to cost-effectively solve the challenges of managing data at remote offices. This paper discusses the issues, requirements, and approaches to effective remote data management, with specific emphasis on remote data protection and backup. Also included is a Best Practices Guide to help assess your remote data management requirements.
BEST PRACTICES GUIDECONTENTSAbstract..................................................................................1Understanding the Challenges of Remote Data.....................1Key Considerations for Managing Remote Data....................1Additional Requirements for Remote Data Backup...............2The Case for Archiving............................................................3The Central Policy/Consolidated Approach to ManagingRemote Data..........................................................................3Disk-to-Disk Consolidated Backup ........................................3Consolidated Archive..............................................................4Managing Remote Data with Adaptec Snap EDR..................4Snap EDR Remote Manager.............................................4Snap EDR Remote Agents................................................5Snap EDR Remote Data Solutions...................................5A Best Practices Guide to Managing Remote Data...............5Summary................................................................................6AbstractThe increasing risk from unprotected user files andremote data (data stored outside the data center) iscausing companies to re-evaluate their current remotebackup processes. Managing remote data poses uniquechallenges given the variability of networks, computingplatforms, lack of trained IT staff at remote locationsand other issues. Further, traditional methods ofmanaging remote data tend to be high cost, unreliable,manually intensive and often require redundantequipment and effort.Advanced remote data management and movementtechnology, such as that incorporated into AdaptecSnap EDR now makes it possible to cost-effectivelysolve the challenges of managing data at remoteoffices. This paper discusses the issues, requirements,and approaches to effective remote data management,with specific emphasis on remote data protection andbackup. Also included is a Best Practices Guide to helpassess your remote data management requirements.Understanding the Challenges of RemoteDataProtecting remote data and managing the exchange ofdata between corporate locations and remote offices isneither trivial, nor cheap.IT administrators in companies with remote officesoften spend significant amounts of their time managingbackup, data management, and data transfer requirements for those offices. Even so, critical processessuch as backups may not be adequately covered. Often,central IT staff must rely on non-technical staff inremote locations to change backup tapes, initiateprocesses and take other actions they are neither trainednor compensated to perform.As a result, companiesreport as much as 60% of their remote backup procedures may fail on a nightly basis. This representsrisk that few companies can afford.When problems do occur, recovery can be tedious andtakes days to recover, assuming the data was adequatelybacked up. Central IT personnel may need to havetapes shipped from the remote site, catalog the volumeand search for the files needed, and then reship thefiles for restore.Online alternatives such as consolidated backup,disk-based backup and centralized archive canincrease reliability and overall data protection,improve speed recovery and have proven to improvereliability, overall data protection and significantlylower costs. These methods are discussed further inthis paper, however there are a number of issues thatmust be considered as you evaluate these newapproaches and the technologies to implement them.Key Considerations for Managing RemoteDataTo effectively managing remote data, you must considerand address a number of specific functional,environmental, and technology factors that do not necessarily come into play in the data center. These factors include:Central Policy-Based Control: To efficiently controldata at remote sites, enterprises must have the abilityto set up and implement central policies. That meanssetting a rule once and having that directive implemented throughout the enterprise, rather thanmanaging activities individually at different sites withmultiple separate platform-specific policies and tools.While many products claim central control capability,they in fact require administrators to establish a uniqueconnection to each remote node to set policy. ThisRemote Data Management & Backup with Snap EDRUntitled DocumentBEST PRACTICE S GUIDERemote Data Management & Backup with Snap EDRapproach eats up hours of administration time, notonly during initial set up, but each time a businessrequirement changes. Some technologies, such asAdaptec Snap EDR, provide a set it and forget it approach that automates the communication of policyto the remote node, and have integrated notification ifsomething does not proceed according to policy.Wide Area Netowrk WAN) Network BandwidthUtilization: Any solution that addresses remote datamust take into account bandwidth restrictions, as wellas a range of network conditions. Remote locationsfrequently have varying bandwidth availability thatneeds to be shared among multiple applications andusers at particular times. For this reason, remote datamanagement and movement solutions should havefeatures that enable efficient use of available bandwidth such as byte-level differential data transfer,bandwidth throttling, multi-streaming, and compression. The amount of network overhead, orinformation that is in addition to the data being transferred, that a product sends over the network is animportant consideration. Obviously, less is better thanmore. Finally, since some remote connections will likelybe impaired during some processes, the ability to restartat the point of failure is critical as well as the ability tore-route information flow to alternate networks.Security and Data Integrity: When moving data overnetworks, data security is always a major concern.Networks are always susceptible to intrusion, but particularly in remote locations where there are fewerIT controls. As a result, any remote data solutionshould authenticate all sending and receiving nodesprior to any data transfer, and encrypt data duringtransmission. Moreover, they should utilize a singlefirewall port and minimize firewall rules. The abilityto ensure that data is received with 100% integrity isalso an important consideration. Where tape is usedfor remote backup, one of the biggest points of recoveryfailure is that data is corrupted on the tape. With somedisk-to-disk backup technologies, data accuracy can be100% guaranteed.Remote Process Automation and Application Interfacing: To minimize or eliminate the need formanual effort at remote locations, the managementsolution must be able to automate processes and interface with remote applications to access data. Forexample, when backing up applications like Exchangeor SQL Server, it is preferable to use native backuproutines. Therefore, the remote data solution must beable to integrate with the application and invoke thenative backup package as part of the backup process.Similarly, for applications such as SAP, data must beaccessed through the application to ensure integrity,instead of accessing it directly at the database,filesystem, or disk levels. In addition, other custom orscript-based processes may also be needed or requiredprior, during or after data transmission. The remotemanagement solution should automate these as partof the overall remote backup process.Heterogeneous System Support: It is common that acompany with multiple remote locations will have avariety of computing platforms and applications at thoselocations.It is therefore important to choose a solutionthat can work within a heterogeneous environment.While this seems simplistic,many products today onlywork within homogeneous environments.Point-in-Time vs. Continuous Replication:Continuous replication products continuously monitor a filesystem and capture changes as they happen and either replicate them immediately orcache the information for bulk transfer at a later time.While these products are ideal for continuous replication between a small number of systems forbusiness continuity purposes, they are not ideal forperiodic processes such as backup and archive. Point-in-time replication products are more appropriate,and in general will be far more network efficient forperiodic processes such as backup and archive.Additional Requirements for Remote DataBackupBeyond the basic remote data considerations listed in theprevious section,there are some specific requirements forremote backup that become important as Consolidatedand Disk-to-Disk Backup methods are considered.Backup at remote offices requires more than just writingthe data to tape.Backup solutions must address dataintegrity and accuracy, automatic operation, offsitestorage, and of course, restoration.When storing user files for backup, it is important thatthe integrity of the original data be preserved; themost important characteristic of a backup is that itcan be restored with full integrity. A backup mustrepresent the true data status at the time of the backup. While tape backup software often has thecapability to handle files left in an open state at thetime of backup, it is important that your disk-backupmechanism have options (skip, open file transfer, orcreate error log) for handling open files.Backup processes for remote offices ideally shouldrequire little or no local manual intervention, butinstead be a completely automated, lights out 2Untitled DocumentBEST PRACTICE S GUIDERemote Data Management & Backup with Snap EDRoperation. At the same time, since offsite data storageis a must, a local backup to tape process at remote sitesmust involve some manual intervention to remove thebackup media to the offsite storage. Online backup,which can transmit data to another location to bebacked up either to disk or to tape, can eliminate theneed for any redundant manual effort at the remotesites.The Case for ArchivingAn often-overlooked, but critical component ofremote data management and protection is archiving.Lets face it, few of us have the time or interest to cleanout our electronic files. Emails building up in Outlookinboxes and other files building up in private andshared directories are contributing to the huge volumeof data growing on remote storage. In a recent surveyby Storage Magazine, users indicated the single biggestreason for backup failure was the quantity of data wastoo large to be backed up within the backup window.The fact is that most user files and email data are seldom re-opened after the first three days ofcreation/receipt. Statistics show that if a file hasn tbeen accessed in 90 days, there s a 90%+ probabilitythat it will never be accessed again. Meanwhile, it consumes valuable storage resources. The problem isthat since we can t predict what 10% we will need, wehold on to all of it. The cost-effective solution is tomove unused data to lower cost secondary storage(archive), with reasonably easy retrieval capabilities,for long-term retention.A second key factor driving the need for archiving isthe federal document regulations, such as SEC Rule17a-4, that require many companies to retain all communication and documentation for specific timeperiods. For a distributed enterprise with manyremote offices, ensuring compliance to these regulations can be a challenge.So, cost and legal requirements are compelling companies to ensure that employee messages arearchived or at the very least, moved to lower-cost,longer-term media. Similar to remote backup, dataintegrity is essential to any archival solution.Consolidated archival, in which older or unused datais automatically moved from remote disks to a centralrepository, often consisting of lower cost ATA disks,while leaving transparent access capability for theremote user, is rapidly gaining acceptance as the mostviable approach to archive for remote data.The Central Policy/ConsolidatedApproach to Managing Remote DataRather than relying on individual backups and separate point processes and staffing required for eachremote site and the staffing required for each, a moreeffective enterprise approach is to allow central IT staffto control remote data management and backup. Thisrequires understanding the changing properties andcharacteristics of remote data. Solutions should beable to set policies pertaining to the data, automateprocesses to execute those policies on remote servers,and be able to move data between remote or edge servers and central or core systems.In this model, individual remote backup and archivingprocesses at the remote sites are replaced with a consolidated process that moves remote data to a hubsite for backup or archive. This requires moving thepertinent data over the available networks in an efficient, secure, timely fashion and therefore requirestechnology that can deal with the many issues associated with controlling and moving data amongmany sites and network connections. These issues areidentified in more detail in the next section.Centrally controlled, automated processes have beenshown to decrease backup costs by as much as 75%due to the elimination of tapes, tape drives, offsite tapestorage, and the redundant staffing efforts at eachlocation.Disk-to-Disk Consolidated Backup Disk-to-disk backup is gaining popularity due to factors including the rapidly falling cost of disk storage, the elimination of the physical limits, and relative unreliability of manual tape storage programs, aswell as the need for more ready access to data for restore.Implemented in a best practices model, disk-to-diskbackup for remote data involves moving the data to bebacked up over a network to a different location. Thereason for this is that disk-to-disk backup, if performedat the same site, does not provide the protection need-ed to recover from a site disaster event (fire, flood, etc.).For any business with multiple remote locations,consolidating disk-to-disk backup brings operationaland cost efficiencies and enhanced data security.There are two primary consolidated backup architectures:" Moving differential data to a central disk " Consolidating backup images Both offer central control and automation and theelimination of individual tapes, tape drives, and offsitetape storage processes at each site.3Untitled DocumentBEST PRACTICE S GUIDERemote Data Management & Backup with Snap EDRIn the first, more common, approach to consolidatedbackup, data at remote sites is periodically analyzed todetermine differential data (i.e. data that has changed)since the last backup process. A copy of this differentialdata is then moved to a central site, where it is storedon disk. Some technologies have the ability to discernjust the byte-level modifications of files to minimizethe amount of data needing to be transferred. Datacan be stored as incremental packets of data or re-constructed on the central site to provide full, up-to-date copies of remote files. This latter alternativeprovides the advantage of offering instant access toindividual files in the case that a remote file is accidentally deleted.In the second approach, backups are run on remoteservers with the output stored to the local disk. Theresulting backup image is then transferred to disk atthe remote site. This works well for applications thathave native backup or snapshot features that can beutilized in the consolidated backup process.These approaches can also be used together. For example, backing up user files may be best performedwith differential data transfer, while backing upExchange data may be best performed using the consolidated backup image approach.In both approaches, the backup data on disk at thecore location can be further sent to tape if desired.Companies often choose to keep one or two days ofbackup data on disk for instantaneous access, with olderdata written to tape.Consolidated ArchiveConsolidated archive involves identifying remote data that meetscorporate archival policy and thenmoving that data from remote systems and archiving it to a centraldisk. Archival policy determineswhat data should be archived andwhen, and parameters often include:last date accessed, type of file,content, location, ownership, size offile, among others. A consolidatedarchival process has many benefitssuch as:" Reducing the amount of data to be backed up(reducing the backup window required)" Optimizing use of remote disk for better performance and cost " Ensuring compliance to data retention policies andregulationsAn essential part of consolidated archive is somemechanism by which data can be retrieved from thearchive by end users, preferably without the involvement of IT.Central policy-based consolidated processes, such asconsolidated backup and archive, provide an approachto managing data at remote offices that can significantlylower costs, eliminate risk, improve data consistency,and also ensure better compliance to corporate backupand retention policies. It is an approach that all businesses with remote offices should actively consider.Managing Remote Data with AdaptecSnap EDRAdaptec Snap EDR is an enterprise-class solution thatenables organizations to centrally control and securelymove data among remote and core locations. Highlyscalable, Snap EDR can handle up to thousands oflocations across heterogeneous Snap Server,Windows,UNIX, and Linux systems. The Snap EDR technologyprovides critical remote data capabilities such as policy-based central control, remote process automation,transport and data-level security, guaranteed dataintegrity, and the ability to deal effectively with manytypes of networks, including high-latency networks.Snap EDR technology is in use by leading companiesworldwide to manage, contro,l and move remote data.Snap EDR ManagerThe Snap EDR Manager is the central control centerfor enterprise-wide remote data processes. Remotedata processes are easily set up, scheduled, deployed.and monitored through the Manager s graphical userinterface.4Untitled DocumentBEST PRACTICE S GUIDERemote Data Management & Backup with Snap EDRSnap EDR AgentsSnap EDR Agents are installed on all Snap Server,Windows, UNIX, Linux or other systems involved inthe remote data processes. They are remotely installedthrough the Snap EDR Manager and execute processesand data transfer based on instructions and rules sentfrom the Manager. They handle multiple tasks, such asdata consolidation, distribution, or synchronizationtasks.Snap EDR Remote Data SolutionsTo make it easier for companies to apply Snap EDRtechnology to solve specific remote data problems,Adaptec has developed a number of solution packagesfor the most prevalent remote data problems. Thesesolutions include:" Remote Data Discovery: To properly manageremote data, a good understanding of data atremote sites is essential. The Remote Data Inventory solution automatically collects andreports on data characteristics at remote locationssuch as file types and sizes, ownership, filecreate/modified/access dates, file system size,utilization, and much more." Consolidated Archive: This solution provides forrules-based archiving of remote data to central systems while providing easy retrieval of thearchived file. Snap EDR will automatically archivefiles based on flexible policies, such as file type, sizeor access date. A unique file marker technologyallows users to retrieve archive files simply by clicking on them." Consolidated Backup: This solution efficientlyconsolidates data from multiple locations for a unified backup process. Snap EDR identifieschanges made to remote files since the last backupand on a scheduled or event-driven basis movesonly the bytes of those files that have changed tothe central site.A Best Practices Guide to ManagingRemote DataBest practices for managing and protecting remotedata involve both understanding and implementingtechnology that supports the remote automatedprocesses. There are five primary steps toward implementing an enterprise-wide remote data management solution:1. Identify and understand remote data and the network environment2. Select a remote data management solution3. Create policy for how remote data should be managed4. Deploy centrally controlled automated processes toimplement the policy5. Monitor and adjust as business conditions changeThe questions below are designed as a guidelinetowards implementing the first two steps of the five-step remote data management process.Assess the current system:" Which types of applications are you using for datatransfers? (e.g. FTP, xcopy, robocopy, tftp, DFS/FRS,public folder replication)" How much manual intervention is currentlyrequired?" What are the failure rates for these systems? Howmuch does it cost when they fail?" How easy is it to adjust to new business requirements?Determine your data movement goals:Knowing your goals for data movement will assist indeveloping the cost-recovery models to justify anypurchases needed. Example goals include:" Reduce backup failure rates/increasing data protection" Reduce meantime to restore for remote office" Ensure regulatory compliance for data archiving" Automate data transfers to and from remote sitesDetermine the types of data that need to be moved:How much data is at the remote locations and whatare the characteristics of the data? (size, file types, diskutilization, etc.)" Which characteristics need to be maintained:i. Ownership preservation?ii. File system attributes?iii. Physical disk layout?What applications are running in the remote locations?" Does my data transfer system need to integrate withparticular vendor applications in real time ?What data are users currently not backing up? (Whatis my current exposure?)What types of data need to be sent to the remoteoffice?" How compressible is this data?Determine the data movement volume:Neither the time available nor the network is infinite.You ll need to crunch the numbers and come up with:5Untitled DocumentBEST PRACTICE S GUIDECopyright 2005 Adaptec Inc.All rights reserved.Adaptec and the Adaptec logo are trademarks of Adaptec, Inc., which may be registered in some jurisdictions.All other trademarks used are owned by their respective owners.Information supplied by Adaptec Inc., is believed to be accurate and reliable at the time of printing, but Adaptec Inc., assumes no responsibility for any errors that may appear in this document.Adaptec, Inc., reserves the right, without notice, to make changes in product design or specifications. Information is subject to change without notice.P/N: 666730-012 Printed in USA 2/05 3704_1.3Remote Data Management & Backup -- Best Practices" What is the rate of change of the data on a day-to-day basis?" How many sites need to be supported?Assess your current network:" What is the available bandwidth to each remotelocation?" What other applications are currently using thisbandwidth? How much bandwidth do these applications require? e.g. Active Directory replication, electronic mail,terminal services" Can traffic be segregated using quality of service(QoS) applications? (i.e. Will you be able to dedicate bandwidth to certain applications?)" Is the network traffic prone to bursts?" How secure is the network? (e.g. Are encryptedVPNs in place to support confidential data transfers?)Choose your solution:" Evaluate potential vendors based against requiredremote data capabilities:" Can the vendor s solution solve multiple remotedata problems, such as backup, and archive and distribution?" Does the vendor have expertise with remote dataapplication and integration?i. Will the vendor assist in assessing your requirements?ii. Will the vendor provide the tools to assess yourdata change and growth rates?SummaryMany companies are re-evaluating their current backup processes, not only to ensure the proper backup of critical data, but also with the goal oflowering overall IT costs, and safeguarding themselvesfrom the penalties of regulatory non-compliance.Managing remote data effectively requires that youdeal with networks variability, dissimilar computingplatforms, security needs, and data integrity, andimplement process automation to overcome the lackof trained IT staff at remote locations.The good news is that all this does not have to be hardor complex. Advanced remote data management andmovement technology such as Adaptec Snap EDR"now makes it possible to cost-effectively solve the challenges of managing data at remote offices with asingle unified approach. Understanding the issues,requirements, and approaches to effective data protection for remote data, with specific emphasis onremote data backup, is the first step to helping yourcompany assess its remote data requirements.For a 3-minute tour of Snap EDR, go tohttp://www.adaptec.com/go/edr_flash/index.html or for more information on Adaptec Snap EDR, visitwww.adaptec.com or call us at 1-800-442-7274.CapabilityVendor 1Vendor 2Vendor 3YesNoYesNoYesNoCentral Policy-BasedControlNetwork EfficiencyRemote ProcessAutomationSecuritySupport forHeterogeneousEnvironmentsPoint-in-TimeReplication6Adaptec, Inc.691 South Milpitas BoulevardMilpitas, California 95035Tel: (408) 945-8600Fax: (408) 262-2533Literature Requests:US and Canada: 1 (800) 442-7274 or (408) 957-7274World Wide Web: http://www.adaptec.comPre-Sales Support: US and Canada: 1 (800) 442-7274 or (408) 957-7274Pre-Sales Support: Europe: Tel: (32) 2-352-34-11 or Fax: (32) 2-352-34-00