HP StorageWorks XP Disk/Cache Partition best practices white paper

An XP12000 disk array can have up to 256 GB of cache memory, over 220 ports, more than 330 TB of internal disk storage, plus the storage capacity of externally attached disk arrays. A disk array of this size and capability can support many applications running simultaneously. However when multiple applications share cache, some applications may consume more than their fair share of cache, causing other applications to be starved of cache memory. This sharing of cache may also pollute the cache tables and degrade the effectiveness of cache memory. For example, a performance mismatch between a fast replication source volume and a slower copy target volume will cause high cache consumption as the writes to the target volume are staged to cache, thus possibly impacting the performance of other applications. Similarly, when applications share disk groups disk-spindle contention can negatively impact performance.

When multiple administrators share the same system, the likelihood that someone might accidentally impact or destroy another administrator’s storage resources or data, or even bring down the entire system, increases dramatically.

XP partitioning protects performance and restricts the scope of damage caused by administrator errors.

HP StorageWorks XP Disk/Cache Partition best practices white paper Executive summary............................................................................................................................... 2 What problems does XP partitioning solve? ............................................................................................ 2 Overview of XP partitioning .................................................................................................................. 3 Cache Logical Partition..................................................................................................................... 3 Storage Management Logical Partition ............................................................................................... 4 XP partitioning details .......................................................................................................................... 5 Concepts ........................................................................................................................................ 5 SLPR/CLPR specifications .................................................................................................................. 7 Shared resources ............................................................................................................................. 7 User roles and permissions................................................................................................................ 8 Planning for partitions ........................................................................................................................ 10 XP logical partitioning use case solutions.............................................................................................. 13 Use case: Protect critical application performance.............................................................................. 13 Advantages ............................................................................................................................... 13 Limitations ................................................................................................................................. 13 Use case: Improve data center security and minimize administrator errors ............................................ 13 Advantages ............................................................................................................................... 14 Limitations ................................................................................................................................. 14 Use case: Partition array between production and development systems............................................... 14 IT challenge............................................................................................................................... 14 Solution configuration ................................................................................................................. 15 Advantages ............................................................................................................................... 16 Limitations ................................................................................................................................. 16 Use case: Isolate external storage cache usage ................................................................................. 16 General XP disk/cache partitioning limitations ...................................................................................... 18 Glossary........................................................................................................................................... 19 Conclusion........................................................................................................................................ 19 For more information.......................................................................................................................... 20  Untitled DocumentExecutive summary The HP StorageWorks XP12000/XP10000 disk arrays and the HP StorageWorks XP Disk/Cache Partition feature provide the ability to subdivide the disk array into secure subsystems and to partition resources so that multiple applications can share the disk array without affecting each other. This white paper describes best use cases and provides configuration recommendations to get the most out the XP Disk/Cache Partition feature. What problems does XP partitioning solve? 1. An XP12000 disk array can have up to 256 GB of cache memory, over 220 ports, more than 330 TB of internal disk storage, plus the storage capacity of externally attached disk arrays. A disk array of this size and capability can support many applications running simultaneously. However when multiple applications share cache, some applications may consume more than their fair share of cache, causing other applications to be starved of cache memory. This sharing of cache may also pollute the cache tables and degrade the effectiveness of cache memory. For example, a performance mismatch between a fast replication source volume and a slower copy target volume will cause high cache consumption as the writes to the target volume are staged to cache, thus possibly impacting the performance of other applications. Similarly, when applications share disk groups disk-spindle contention can negatively impact performance. 2. When multiple administrators share the same system, the likelihood that someone might accidentally impact or destroy another administrator s storage resources or data or even bring down the entire system increases dramatically. XP partitioning protects performance and restricts the scope of damage caused by administrator errors. 2 Untitled DocumentOverview of XP partitioning Cache Logical Partition The Cache Logical Partition (CLPR) feature allows the XP12000/XP10000 disk array cache memory to be partitioned into multiple virtual cache memories to provide cache performance protection. In addition to the partitioning of cache memory, disk groups are also assigned to a particular CLPR, providing disk-spindle performance isolation (Figure 1). The CLPR feature protects against cache competition and insulates applications from potential performance degradation caused by other applications. The CLPR feature can also be used to isolate high cache-consuming replication target volumes.  Figure 1. CLPRs            Subsystem Legend: : Host I/O Storage administrator Cache Memory (128 gigabytes) CLPR1 (40 gigabytes) Parity group 1-1 : AdministrationHost of Branch AParity group 1-2 Parity group 1-3 CLPR2 (40 gigabytes)  CLPR3 (40 gigabytes) Host of Branch BHost of Branch C   3 Untitled DocumentStorage Management Logical Partition The Storage Management Logical Partition (SLPR) feature allows the XP12000/XP10000 disk array subsystem to be divided into multiple virtual disk arrays and provides a rich set of user roles and permissions. One or more CLPRs and CHIP ports are then assigned to the SLPR. Each SLPR can be assigned a Storage Partition Administrator (PSA) to manage the virtual disk array formed by the SLPR. A PSA can only access specifically assigned SLPRs and their resources, and cannot access other SLPRs. The SLPR user role model provides increased security for user data and XP resources, and limits the scope of administrator errors to a single partition. In figure 2, below the left side illustrates a PSA managing an SLPR and the isolation of resources in two CLPR. The right side illustrates 2 SLPRs, each managed by different PSAs and the privacy of those resources to each SLPR.  Figure 2. SLPR administration model CacheParity GroupParity GroupCacheFC PortsCacheParity GroupParity GroupCacheFC PortsFC PortsCacheParity GroupParity GroupCacheFC PortsCacheParity GroupParity GroupCacheFC PortsFC Ports One person managing a SLPR with two CLPRs; the CLPR  Two SLPRs, each managed by different admins resources are insulated from one another  4 Untitled DocumentXP partitioning details Concepts CLPR 0 is predefined as the initial pool of cache and disk groups and SLPR 0 is predefined as the initial pool of ports and CLPR 0. CLPR 0 is always assigned to SLPR 0, and neither CLPR 0 nor SLPR 0 can be removed. The Storage Administrator (SA) manages the array and can define up to 31 additional SLPRs and can also create up to 31 CLPRs for a maximum of 32 SLPRs and 32 CLPRs. Figure 3 shows several examples of how SLPRs, CLPRs, and ports can be arranged. Each SLPR is assigned one or more CU and SSIDs, and no two SLPRs can share CU/SSIDs (Figure 4).  Figure 3. XP logical partition concepts SLPR2SLPR3SLPR0CLPR2CLPR3CLPR4CLPR0(pool)CLPR5SLPR0 CLPR0 (pool)PortsSLPR0 CLPR1CLPR2CLPR3CLPR4CLPR0 (pool)PortsNo user defined partitionsCLPR onlySLPR1CLPR1SLPR and CLPRSLPR2SLPR3SLPR0CLPR2CLPR3CLPR4CLPR0(pool)CLPR5SLPR0 CLPR0 (pool)PortsSLPR0 CLPR0 (pool)PortsSLPR0 CLPR1CLPR2CLPR3CLPR4CLPR0 (pool)PortsNo user defined partitionsCLPR onlySLPR1CLPR1SLPR and CLPR   5 Untitled Document Figure 4. XP logical partition relationships SLPR  " CLPR contains- Specific cache capacity- One or more Parity Groups" SLPR contains- CLPRs- PortsParity GroupABCDCache MemoryParity GroupPort&.&.&.Physical ConfigurationCLPRPartitionedcachessignedParity GroupssignedCLPRssigned&.Port&.xGB&.6 Untitled DocumentSLPR/CLPR specifications Table 1 summarizes some of the SLPR and CLPR characteristics. Table 1. SLPR/CLPR specifications SLPR/CLPR specifications XP12000 XP10000 Maximum SLPRs per array 32 32 Maximum CLPRs per array 32 16 Maximum CLPRs per SLPR 32 16 Maximum CLPR cache capacity 256 GB 64 GB Minimum CLPR cache capacity 4 GB 4 GB Minimum cache memory change unit 2 GB 2 GB Minimum disk groups per CLPR 1 1 Maximum VDEV per CLPR 16384 16384 VDEV change unit 1 1 Emulation types All XP12000 supported emulation types All XP10000 supported emulation types LUSE support Yes Yes Cache LUN support Yes Yes PCR support Yes Yes RAID configurations All XP12000 supported RAID configurations All XP10000 supported RAID configurations  Shared resources While the XP Disk/Cache Partition feature logically partitions cache memory, segregates disk groups, and allows you to divide the subsystem into separately managed virtual subsystems, the following array resources cannot be partitioned: " ACP processors " Batteries " FC-AL paths " Cache switches " Control paths " Data paths " Backend loops ACP processors and HDD backend loops cannot be partitioned and are shared by all CLPRs. However, through careful planning ACP processors and backend loops can be segregated so that those resources can be focused to a particular CLPR. 7 Untitled DocumentUser roles and permissions The XP12000/XP10000 Cache/Disk Partition feature includes new user roles and permissions to provide the enterprise with a rich and flexible administration model. The following list describes the user roles and their permissions: 1. Storage Administrator with super-user permissions (SA)  Modify entire array configuration Create users and administer partitions Configure XP Remote Web Console settings 2. Storage Administrator with general permissions (GSA)  Modify entire array configuration 3. Storage Partition Administrator with super-user permissions (PSA)  Access to assigned partitions only Modify partition configuration Create and manage other users for assigned partitions only 4. Storage Partition Administrator with general permissions (GPSA)  Access to assigned partitions only Manage assigned partitions The user entry panel (Figure 5) provides the super-user administrator with fine-grain control of feature and resource access. 8 Untitled Document Figure 5. User entry panel   9 Untitled DocumentPlanning for partitions 1. Determine application I/O patterns. "    Use either host-based tools such as sar or perfmon or use Performance Advisor XP to determine the I/O rate or the throughput. "    Determine the I/O block size using sar or perfmon. "    Determine the Read/Write ratio. 2. Estimate cache to disk requirements. Table 2 summarizes some application types and their typical I/O pattern and provides some guidelines on their cache memory demands. For disk groups where the workload is random I/O, use Table 3 to estimate the standard cache capacity (cache memory size). Table 2. Application/cache recommendations Application type I/O pattern %R/%W Block size Cache demand Cache size recommendation Recommendation OLTP Random 60/40 Small High Low Med High writes will consume cache: Data warehouse Sequential 90/10 Small High Low Full table scans use sequential pre-fetch, which caches d 32 MB Backup source Sequential 100/0 Large Low Low Full table scans use sequential pre-fetch, which caches d 32 MB Backup target Sequential 0/100 Large Low Low Sequential pre-fetch, which caches d 32 MB Exchange mail Random 60/40 Small High Med High Write bursts should have a large cache  Table 3. Standard cache capacity  Capacity for the Data in CLPR to 0.7 TB to 2.6 TB to 8.5 TB to 14.4 TB to 20.3 TB 20.3 TB or more Standard Cache Capacity 4 GB 8 GB 12 GB 16 GB 20 GB 24 GB  10 Untitled DocumentWhen calculating the cache capacity for a CLPR, you should take into consideration any cache memory that you want to allocate for use by Cache LUN XP (DCR or Cache Residence) and by NAS (Partial Cache Residency, PCR). Cache capacity is calculated with the formula: CacheCapacity = StandardCacheCapacity + CacheLunSize + PartialCacheResidency " StandardCacheCapacity (GB): The expected allocation of parity group disk capacity that will be used, for example, data. " CacheLunSize (GB): A feature to permanently cache a LUN or a portion of a LUN. Default value is 0 GB. " PartialCacheResidency (GB): Related to NAS use of LDEVs in parity group. Default value is 0 GB. Example: " An application will require 2 TB of disk space and has a small block random R/W IO pattern. StandardCacheCapacity: 8 GB " No plans to permanently cache any LUNs: StandardCacheCapacity: 0 GB " No NAS use of LDEVs in parity group: PartialCacheResidency: 0 GB CacheCapacity = 8 GB + 0 GB + 0 GB = 8 GB 3. Determine license capacity requirements. When you create a Storage Management Logical Partition (SLPR), the partition license keys are disabled by default. Use the License Key Partition Definition tab, located on the Install pane, to enable license keys for the new partition. Basic license keys include XP Web Console and Java" API. Both are required to enable the Web GUI for new partitions. License capacity is assigned to each SLPR based on the usable capacity of all assigned CLPR disk groups. For example, if the total license capacity for LUN Manager is 80 TB and the Storage Administrator (SA) has created SLPR 1, CLPR 1 with 10 TB and SLPR 2, CLPR2 with 20 TB, leaving SLPR 0, CLPR 0 with 50 TB, then the SA should allocate 10 TB of LUN Manager license capacity to SLPR 1 and 20 TB of LUN Manager license capacity to SLPR 2. Figure 6 shows that the product license keys for HP LUN Config & Security Mgr XP Volume Mgr have been allocated to SLPR name SAP R3. 11 Untitled Document Figure 6. License key partition definition panel   12 Untitled DocumentXP logical partitioning use case solutions Use case: Protect critical application performance When application response times are critical to maintaining SLAs, you can isolate performance-critical applications in their own CLPR. CLPRs can minimize negative performance impacts from other applications that share the array. This allows you to focus the required cache memory and disks to critical applications. Advantages " Cache and disk resources focused on a performance-critical application " Guarantee that the application has the cache resources " Cache performance isolation from competing applications " Prevention of cache table pollution to improve cache hits " Disk-spindle performance isolation Limitations " Not all application I/O patterns benefit from extra cache memory. " Each CLPR definition requires a minimum of 4-GB cache memory and one or more parity groups. Use case: Improve data center security and minimize administrator errors 1. Protects multiple administrators from impacting or destroying each others  storage resources and data. "    Partitions can be managed by a partition administrator and any errors or mistakes that the partition administrator makes are confined to his or her partition. "    Other partitions and global resources are protected. 2. Delegated storage management Management improved in that the resources assigned to a partition can be safely managed by a partition administrator. To subdivide the array: 1. Subdivide the array into virtual disk arrays by a creating SLPR for each virtual array. 2. Assign host target ports to each SLPR. 3. In each SLPR create CLPRs, each with the required cache memory and disk groups. 4. Create a PSA user account and assign specific SLPRs to it. 5. Based on each SLPR/CLPR TB capacity assign sufficient feature license capacity to each SLPR, for example, LUN Manager, Auto LUN, and so on. 13 Untitled DocumentAdvantages " SLPR Partition Storage Administrators have the authority to safely administer their respective resources, including LUN management, LUN security, and port management. " The scope of SLPR Partition Storage Administrator errors is confined and reduced to the resources of a single SLPR partition. " Allows for concurrent administration of SLPRs by the respective SLPR Storage Partition Administrators. " A SLPR Super Partition Storage Administrator (SPSA) can create General Partition Storage Administrator users to assist with SLPR resource management. " SLPR Partition Storage Administrator view is restricted from viewing all but assigned resources. " Host target ports are securely isolated to the assigned SLPR. Limitations " When the Storage Administrator is making configuration changes to SLPR[n], the Partition Storage Administrators cannot be logged in to the Remote Web Console (RWC). " SLPR Partition Storage Administrator is restricted from managing replication features, for example, HP StorageWorks Continuous Access XP. " Each SLPR must have sufficient license capacity. Use case: Partition array between production and development systems This use case examines the consolidation of production and test/development storage resources onto an XP12000 disk array. IT challenge Consolidation addresses the issues of lowering costs and increasing the effectiveness of data center resources. There are, however, some key concerns associated with consolidating discreet storage systems onto a single platform: " The IT department must meet its production application SLAs to effectively serve the company s end customers. " Configuration change policies for the production and for the test/development environments must be followed. The production environment policies are more restrictive than the experimental test/development environment, which requires agility and flexibility. 14 Untitled DocumentSolution configuration The XP12000 array is configured with 32-GB cache memory, 24 FC ports, and 16 parity groups. One SLPR/CLPR combination is created for the production environment and another SLPR/CLPR is created for the test/development environment. The XP12000 array resources are divided based on the needs of the two environments. To meet the required SLAs, the production SLPR is allocated 16 host target ports, 16 GB of XP cache memory, and eight parity groups. The test/development SLPR is allocated four host target ports, 4 GB of cache memory, and four parity groups. The four ports will remain in SLPR0 and 4 GB of cache memory; two parity groups will be left in CLPR 0. Figure 7 depicts how the server and XP12000 array partitions can be configured to meet the IT department s production and test/development resource and priority requirements.  Figure 7. XP12000 production/development partitions XP12000Dev-SLPRDev-CLPRCache(4 GB)Target PortsSLPR0CLPR0Cache (4 GB)Prod-SLPRProd-CLPRCache (16 GB)Target PortsXP12000Dev-SLPRDev-CLPRCache(4 GB)Target PortsDev-SLPRDev-CLPRCache(4 GB)Target PortsSLPR0CLPR0Cache (4 GB)SLPR0CLPR0Cache (4 GB)Prod-SLPRProd-CLPRCache (16 GB)Target PortsProd-SLPRProd-CLPRCache (16 GB)Target Ports  15 Untitled DocumentResource partitioning To partition the XP12000 array to meet the solution requirements, the SA creates a production SLPR and a test/development SLPR. To each SLPR, the SA adds the required host target ports and a CLPR. To each CLPR the SA assigns the required cache memory and disk parity groups. Finally, the SA allocates feature licenses and assigns a PSA to each SLPR to securely manage the resources. Advantages " Supports storage consolidation. " When development and testing winds down, XP resources can be re-assigned from the development partitions to the production partitions. " Storage logical partitions free the root SA from having to perform tedious LUN management and provisioning tasks. The SA is free to focus on other more strategic tasks, including planning, replication, and performance analysis. " SLPR Partition Storage Administrators have the authority to safely administer their respective resources, including LUN management, LUN security, and port management. " The development team has the flexibility to independently make LUN provisioning changes without affecting production systems. " Cache, parity groups, and host target ports can be dedicated to an application. Because cache can be dedicated to a specific application, an application s cache performance will be unaffected by the activities of other applications. " Storage partitions are secure and can be independently managed, limiting administrator changes and errors to a single partition. " Increased flexibility to reallocate resources as needs change over time. Limitations " Partitioning cache for dedicated use by specific applications may require more cache than a shared cache model. " Each CLPR requires a minimum of 4 GB of cache memory. " Cache resource allocation requires manual configuration. Currently, there are no built-in facilities for dynamically adjusting cache resources to meet the application load requirements. " PSAs have limited XP feature access. There is no access to HP StorageWorks Business Copy XP, System Status, and so on. " Some XP resources remain global and cannot be partitioned, such as ACPs, CHIPs, control and data paths for cache switches and shared memory, and back-end HDD FC-AL paths. " HP StorageWorks Continuous Access ports and HP StorageWorks External Storage ports are always assigned to SLPR0. Use case: Isolate external storage cache usage With the HP StorageWorks External Storage XP product you can host XP12000/XP10000 disk array datasets on select external storage subsystems, including the HP StorageWorks Modular Smart Array 1000 (MSA1000)/1500 (MSA1500); HP StorageWorks XP48, XP128, XP512, and XP1024 disk arrays; and certain third-party arrays from IBM, EMC, Hitachi, and Sun. While the external storage feature is meant for use cases such as migrating data from other vendor s arrays or older arrays onto the XP, moving less frequently accessed data to less expensive HDDs and for extending XP replication services to subordinate arrays there are some things to consider. The following are some general guidelines. For a detailed description of external storage guidelines configuration rules, refer to the HP StorageWorks External Storage XP user guide. 16 Untitled DocumentRelocate LUNs/volumes from CLPR0 when any of the following conditions exist: " An external storage volume is the copy target of a data mirroring product, for example, BC, CA, P-VOL, S-VOL, Snapshot pool. " External storage LUN cache property is enabled. " An external storage volume is the copy target of a data migration product, for example, Auto LUN XP. " An external volume is the copy target of a cached LUN source volume. 1. Determine the number of CLPR partitions required. a.   Do not mix internal and external LUNs in the same CLPR. b.   Group arrays with similar performance and availability characteristics into the same CLPR. c.   Group external storage LUN BC, CA copy target volumes together into the same CLPR. d.   Group external storage LUN Snapshot primary and external storage pool volumes together into the same CLPR. e.   Put external storage LUNs that have the cache property enabled in their own CLPR. 2. Determine cache memory size of each CLPR. Size the cache according to the degree of locality and the expected performance enhancement. 3. In SLPR 0, configure ports to connect to the subordinate array system as initiator ports. 4. Based on steps 1 and 2, create CLPRs in SLPR0 and assign cache memory. An advantage of relocating external storage volumes out of CLPR0 is the protection from cache competition.   17 Untitled DocumentGeneral XP disk/cache partitioning limitations " Partitions are logical, not physical in that they are implemented by microcode address barriers. " Each CLPR definition requires a minimum of 4-GB cache memory and at least one disk group. " You cannot change the capacity value of CLPR0 directly. If you change another CLPR s capacity, the difference is reflected in CLPR0 s capacity. " There are no built-in facilities to automatically adjust cache memory allocation. " CLPR[n] resources cannot be changed by the Storage Administrator while the PSA[n] is logged in to the RWC. " Each partition requires sufficient license capacity. " PSAs have limited XP feature access. Refer to HP StorageWorks XP Remote Web Console User Guide for XP12000/XP10000, Table 3-8. " Time required to re-assign cache memory is approximately 5 min/GB or can be blocked if writes pending are greater than or equal to 60% or there exists a pinned track condition. " When cache memory is being re-assigned between CLPRs, the cache that is being moved is not available to any partition from the start of the move until the cache is re-assigned to the new partition. " Not all application I/O patterns benefit from cache memory. " Continuous Access ports must belong to SLPR0. " External Storage ports must belong to SLPR0. " Mainframe volumes must belong to CLPRs in SLPR0. " The following internal XP resources are always shared and cannot be assigned to a partition: Shared memory ACP processors Batteries FC-AL paths Cache switches Control paths Data paths Backend loops 18 Untitled DocumentGlossary These terms used throughout this white paper aid in understanding the innovative solutions provided by HP servers and storage. Term Definition Array Group See RAID group CHA Client Host Adapter CHIP Client Host Interface Processor CLI Command Line Interface CU Control Unit CLPR Cache Logical Partition DCR Data Cache LUN Residence Disk Group See RAID Group FC Fibre Channel SSID Subsystem ID GUI Graphical user interface HA High-Availability HBA Host bus adapter IOPS I/Os per second LDEV Logical device LUN SCSI Logical Unit Number nPar HP nPartition (hard partition) OLTP Online Transaction Processing Parity group See RAID group PCR Partial Cache Residence PSA Partition Storage Administrator QoS Quality of Service RAID group The set of disks that make up a RAID set RWC Remote Web Console SA Storage Administrator SAN Storage area network SLA Service Level Agreement SLPR Storage Management Logical Partition SPOF Single Point of Failure WWN World Wide Name, a unique 64-bit device identifier in a Fibre Channel storage area network Conclusion The HP StorageWorks XP12000/XP10000 disk arrays and the HP StorageWorks XP Disk/Cache Partition feature protects performance, minimizes damage scope from mistakes, and provides solutions for many use cases. 19 Untitled DocumentFor more information For additional information, refer to the following resources. Table 4. Resources Resource description Web address HP StorageWorks XP Disk/Cache Partition Overview & Featureshttp://h18004.www1.hp.com/products/storage/software/xpdisccp/index.htmlHP StorageWorks XP Disk Array Family Softwarehttp://h18000.www1.hp.com/storage/xparraysoftware.htmlHP StorageWorks XP10000 Disk Array Manuals (guides, supplements, addendums) HP Business Support Centerhttp://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&locale=en_US&docIndexId=179911&taskId=101&prodTypeId=12169&prodSeriesId=1099402HP StorageWorks External Storage Softwarehttp://h18004.www1.hp.com/products/storage/software/extstxp/qa.html?jumpid=reg_R1002_USENHP StorageWorks XP12000 Disk Array Manuals (guides, supplements, addendums) HP Business Support Centerhttp://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=179911&taskId=101&prodTypeId=12169&prodSeriesId=436460HP StorageWorks Services http://www.hp.com/hps/storage       2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. ava is a U.S. trademark of Sun Microsystems, Inc. 4AA0-6542ENW, August 2006 20