White PapersSIP with its IMS extension is a key component of IMS building blocks such as Call Session Control Function (CSCF), Media Resource Function (MRF), Application Server, User Equipment and Gateways. The IMS performance characterization strategy is to benchmark basic SIP primitive message processing in a typical call setup and teardown scenario.
System Under Test (SUT) is configured as a stateful SIP proxy between SIP clients representing the call load. The call load is provided from a commercially available off-the-shelf SIP call tester known as ProLab* Test Management Suite supplied by Radvision.
White PaperSet Up Procedure for PerformanceCharacterization Results: Benchmark to Demonstrate Improvement in SIP Performance for IMS, Powered by Dual-Core Intel Xeon ProcessorsUntitled DocumentWhite Paper Set Up Procedure for Performance Characterization Results2Table of Contents1.0 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32.0 Annex A: SIP Performance Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42.1 SIP Performance Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42.1.1 Performance Characterization Disclaimer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42.1.2 Performance Characterization Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42.1.3 Test Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42.1.4 Test Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53.0 Annex B: Systems Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83.1 SBC Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83.2 Load Generating Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8Untitled DocumentIntroductionThis document provides the detailed technical informationnecessary to understand test parameters and results characterizing single board computer (SBC) performance for IMS workload based on SIP transactions.As an example, the benchmark results for the Intel NetStructure MPCBL0040 Single BoardComputer can be found at http://www.intel.com/design/telecom/papers/9936.htm3Set Up Procedure for Performance Characterization Results White PaperUntitled DocumentWhite Paper Set Up Procedure for Performance Characterization Results42.0 Annex A: SIP PerformanceCharacterization2.1 SIP Performance Characterization2.1.1 Performance CharacterizationDisclaimerSIP testing was performed to determine a relativecomparison of single board computer (SBC) performance reflecting the performance of themicroprocessor, memory architecture, and compilerof a computer system on compute-intensive, 32-bitapplications. The test results were determinedusing specific, well-configured systems. Theseresults may or may not reflect the relative performance of an SBC in systems with differenthardware, software designs or configurations(including compilers). Buyers should consult othersources of information, including system bench-marks, to evaluate the performance of systemsbeing considered for purchase. The production and pre-production SBC performancemeasurements disclosed in this document wereconducted during the period of September toDecember 2005.2.1.2 Performance CharacterizationDescription2.1.2.1 Test StrategySIP with its IMS extension is a key component ofIMS building blocks such as Call Session ControlFunction (CSCF), Media Resource Function (MRF),Application Server, User Equipment and Gateways.The IMS performance characterization strategy is tobenchmark basic SIP primitive message processingin a typical call setup and teardown scenario. System Under Test (SUT) is configured as a statefulSIP proxy between SIP clients representing the callload. The call load is provided from a commerciallyavailable off-the-shelf SIP call tester known as ProLab*Test Management Suite supplied by Radvision. This test measured the processor usage of theSUT across a range of calls per second. During thetest, the number of calls per second (CPS) wassteadily increased as the SUT processor utilizationand average call setup time performance wasmeasured. To ensure the SIP stack was operatingcorrectly, the average call set up time was monitoredto confirm that calls were being answered in atimely manner or in less than 32 seconds.Increasing the CPS workload beyond the capacityof the SUT eventually resulted in timeout messageswhich led directly to failed calls. When the failedcall volume reached 0.5% of total call volume,tests were stopped and the performance boundarywas determined in terms of maximum Busy HourCall Attempts (BHCA) (CPS x 3600) and percentageof CPU utilization. SIP Scenario Setup: " Call Duration: One second" SIP messages: Call Setup: INVITE, 180 Ringing, 200 OK and ACK Call Tear down: BYE and 200 OKNote: Under the maximum system load, averagecall setup time can reach 300 ms. Call hold durationof one second was selected for this test as thevalue needed to be greater than the worst casecall setup time of 300 ms. 2.1.3 Test Setup2.1.3.1 Hardware" GbE switch to connect SBC with SIP sessionload generating modules " SBC to be benchmarked" Multiple SIP session load generating modules See Annex B for details of the SBC configurationsthat were tested.Untitled Document5Set Up Procedure for Performance Characterization Results White Paper2.1.3.2 Software" Stimulus Application: Radvision ProLab Test ManagementSuite generated the calls and managed the test scenario." Application Under Test: Radvision SIP Server Toolkit*(version 2.5) compiled under Linux (RedHat AS 4.0) usingGNU C compiler (version 3.2.3) was chosen as the SIP stack.This toolkit was a complete framework for developing alltypes of SIP Server applications including proxies, redirectservers, registrars, presence servers and different types ofB2BUAs and 3rd party call control. The toolkit was suppliedas source code which needed to be compiled for the targetplatform. The SIP Server Toolkit was compiled using theGNU compiler suite as commonly used within the industry. GNU 3.2.3 compiler options used:CC := gccCFLAGS += -Wall -W -DRV_TOOL_STRING= (RV_TOOL_STRING)C_OPTFLAG := -O22.1.3.3 Modules and Roles" SIP Caller: SIP endpoint represented by Radvision ProLabTest Management Suite. Initiated bulk calls to the receivervia the SIP Server." SIP Callee: SIP endpoint represented by Radvision ProLabManagement Test Suite. Received bulk calls." SIP Stateful Proxy: Radvision SIP Server Toolkit runningon the SUT.SIP CallerProlabToolSIP CalleeProlabToolSIP Stateful Proxyrunning on SystemUnder Test2.1.4 Test DetailsThe testing approach included the derivation of a set ofparameters from the SUT when the SIP Server applicationwas under incremental set of CPS load. Consequent graphsand analysis were then created using these parameter sets.The parameters were as follows:" CPU Usage (User): Average CPU usage in the user spaceunder a particular SIP call load. " CPU Usage (System): Average CPU usage in the systemspace under a particular SIP call load." CPU Usage (Total): Average total CPU usage, or user andsystem space CPU usage, under a specific SIP call load." CPS: Average SIP CPS handled by the SIP Server." Average Call Setup Time: Average time required to setupa SIP call under a specific SIP call load. ProLab TestManagement Suite reported average call setup time." BHCA: Average BHCA generated towards the SIP Server.BHCA value was calculated as CPS x 3600.Test Setup FigureUntitled DocumentWhite Paper Set Up Procedure for Performance Characterization Results6The following table illustrates the call flow and the SIP messages that are involved in a single call. The callduration was one second. This scenario was repeated for bulk calls under a Poisson distribution. TheProLab SIP Test Agent (side A) made bulk non-media calls to another ProLab SIP Test Agent (side B) viathe SIP Server toolkit. Side A (Sender)SIP Server (SUT)Side B (Receiver)SIP end point waits for a callSIP agent makes a callReceive from Side A: INVITEReceive call; send ringingReceive from Side B: 180 ringingSend to Side A: 180 RingingReceive ringing messageAnswer the callReceive from Side B: 200 OKSend to Side A: 200 OKReceive call answeredmessage; Send ACKReceive from Side A: ACKSend to Side B: ACKReceive ACKSIP agent hangs upReceive from Side A: BYESend to Side B: BYECall disconnected; Send OKReceive from Side B: 200 OKSend to Side A: 200 OKCall disconnectedUntitled Document7Set Up Procedure for Performance Characterization Results White PaperCPSBHCACPU Usage%CPU Usage%CPU Usage%Average Call(User)(System)(Total)Setup (ms)100360,00003.90.704.615.3200720,00007.81.209.015.53001,080,00012.73.115.815.54001,440,00016.42.518.915.75001,800,00020.42.923.317.26002,160,00026.84.331.122.07002,520,00031.25.336.530.08002,880,00035.56.041.535.09003,240,00040.76.747.438.010003,600,00044.47.551.945.111003,960,00049.98.458.363.012004,320,00054.78.763.467.013004,680,00059.59.268.771.014005,004,00064.79.674.396.415005,400,00070.411.081.4115.016005,760,00074.310.684.9116.017006,120,00078.412.190.5198.0Performance Boundary: Beyond 1700 CPS and 90.5% total CPU Usage the SIP Server was unable to handle the calls in theappropriate time (32 sec) and the calls received timeout messages leading to failed calls. These measurements were takenwhen failed calls were less than 0.5% of the total calls.The following table shows an example of test measurements from using the Intel MPCBL0040 SBC as the SUT. Untitled DocumentWhite Paper Set Up Procedure for Performance Characterization Results83.0 Annex B: Systems Configurations3.1 SBC ConfigurationsThe following table summarizes the SBC configurations that were tested during the SIP performancecharacterization.3.2 Load Generating Modules Test setup included multiple SBCs running the Radvision ProLab Test Management Suite to generate thecalls. The number of load generating modules required will vary based on load generating software aswell as performance capacity of system under test. Our test setup used eight Dual-Core Intel Xeon 2GHzprocessors to generate the SIP workload. Intel NetStructure SBCIntel MPCBL0001Intel MPCBL0040(N04 or F04 SKU)Processor Type/NameLow Voltage Intel Dual-Core Intel Xeon Xeon Processorprocessor LV 2.0 GHzProcessorsTwoTwoDual CoreNoYesHyper-Threading Technology SupportYesNoProcessor Frequency2.0 GHz2.0 GHzProcessor Bus Speed (PSB)400 MHz667 MHzChipsetIntel E7501 Chipset Intel E7520 and Intel I/O Controller Chipset and Intel 6300ESBHub 3I/O Controller HubMemory BusDDR-200/266DDR2-400 Memory Quantity4GB4GBOSRedHat* AS 3.0 Update 3RedHat AS 4.0Local Storage60GB40GBBase InterfaceDual GbE port Dual GbE ports (Intel 82546EB) (Intel 82571)Fabric InterfaceTwo Fibre Channel* (F04 Four GbE (Intel 82571) SKU only) QLogic* ISP2312 ports, default connection to Fabric interfaceFibre ChannelYes (F04 SKU only)NoUntitled DocumentINFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCHPRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND/OR USE OF INTEL PRODUCTS, INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT, OR OTHER INTELLECTUAL PROPERTY RIGHT. Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measuredby those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of informationto evaluate the performance of systems or components they are considering purchasing. For more information on performance tests and on the performance of Intel products, visit http://www.intel.com/performance/resources/limits.htm.Intel Corporation may have patents or pending patent applications, trademarks, copyrights, or other intellectual property rights that relate to the presented subject matter.The furnishing of documents and other materials and information does not provide any license, express or implied, by estoppel or otherwise, to any such patents, trademarks,copyrights, or other intellectual property rights. Intel products are not intended for use in medical, life-saving, life-sustaining, critical control or safety systems, or in nuclear-facility applications. Intel may make changes tospecifications and product descriptions at any time, without notice.Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future definition andshall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them.This document and the software described in it are furnished under license and may only be used or copied in accordance with the terms of the license. The informationin this document is furnished for informational use only, is subject to change without notice, and should not be construed as a commitment by Intel Corporation. IntelCorporation assumes no responsibility or liability for any errors or inaccuracies that may appear in this document or any software that may be provided in association withthis document. Except as permitted by such license, no part of this document may be reproduced, stored in a retrieval system, or transmitted in any form or by any meanswithout the express written consent of Intel Corporation. Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or by visitingIntel's website at http://www.intel.com.Intel, Intel logo, Intel. Leap Ahead. Logo, Intel NetStructure, Pentium and Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in theUnited States and other countries. Hyper-Threading Technology (HT Technology) requires a computer system with an Intel Pentium 4 processor supporting HT Technology and an HT Technology-enabledchipset, BIOS, and operating system. Performance will vary depending on the specific hardware and software you use. See http://www.intel.com/info/hyperthreading/formore information including which processors support HT Technology.*Other names and brands may be claimed as the property of others.Copyright 2006 Intel Corporation. All rights reserved.www.intel.com
