performance test performance test is through a variety of automated testing tools simulate customary and eccentric peak system load conditions to test the performance index. Load testing and stress testing are always performance tests, the two can be in association. By load testing to determine the workload in a variety of system performance, the target is to gradually increase the load test, the system changes the performance indicators. Stress testing is a system by identifying bottlenecks or can not receive performance points, to get the system to provide maximum service level of the test.
directory
Application performance testing the contents of the performance of the consumer test applications on the network performance testing application performance testing on the server the purpose of performance testing the type of performance test performance test performance observed indicators testing steps to set goals and analysis system elect the test method of measurement of learning-related technologies and tools to develop evaluation criteria designed to run test cases test cases test the performance of benchmark test method performance arranging Ceshi peak penetration testing testing performance testing tool automated testing introduction Conclusion chapter describes tools LR test performance test performance test of the content type of performance testing in software quality assurance activities an major role, which includes the test content wealthy and varied. China Software Testing Center will be summarized in three aspects of performance testing: performance of the client application testing, application performance testing on the network and application performance test on the server side. Typically, three effective and reasonable composition of system performance can be achieved on the comprehensive analysis and bottleneck prophecy. Application Performance Testing in the client application on the client side performance testing intentions to examine the performance of client application to test the entry is the client. It includes concurrent performance test, fatigue test, a crowd of of data such as test and speed test, performance testconcurrent performance of which the image
testing is opener. Concurrent concurrent representation testing is the opener process of performance testing is a load testing and tension testing process, that is, increasing the load until the system tin no receive the performance obstruction or point of transaction through a comprehensive analysis of performance indicators and monitoring indicators to resolve the system resource concurrency performance of the process. Load test (Load Testing) is to decide the workload in a variety of system performance, the target is apt gradually boost the load test, the system components of the corresponding output items, such as throughput, answer time, CPU load, memory method, etc. to make sure system performance. Analysis of load testing is a software petition and patronize structure, a virtual reality context, thereby to determine the performance of the process can receive. Stress test (Stress Testing) is a system at identifying bottlenecks or can not receive performance points, to get the system to provide maximum service class test. The purpose of concurrent performance testing is reflected in three aspects: the real business based above a representative selection, the key operational design test cases to reckon the system's present performance; while the expansion of petition functionality or current applications procedure will be deployed, the load test will aid determine whether the system is also proficient to handle the anticipated user load, to portend the future performance of the system; by simulating hundreds of users, and sprint reiterated tests to confirm the performance bottlenecks and optimize and correction application, the purpose is to detect the bottlenecks. When the strength of their organization or a company commissioned to amplify a software application corporation for the period, particularly since the actual use in a creation context, users ambition tend to doubt that this system can resist a massive digit of simultaneous users synchronous access? This The most common kind of problem with online transaction processing (OLTP) database application means, Web browsing and video above demand systems. The problems with the assistance of technological software testing tools and progressive test tools. For example: telecom billing software is well understood approximately 20th of every month if payout is the city's tip, the city's thousands of charging locations same time. Fare collection process is generally divided into 2 steps, first to the user's tel number to check out the monthly costs incurred, and then receive cash and pay the consumer adjust the state has. A user looks uncomplicated 2 steps, yet while hundreds of thousands of extremities, meantime the implementation of such actions, the location is quite differ, so numerous transactions happen simultaneously, the application itself, the operating system, the central database waiter , middleware servers, web equipment, affordability is a caustic test. Decision-makers can not be a problem before thinking the affordability of the system, predicted tolerance of concurrent software, which is in the software testing period should be determined. Currently, most companies need to support hundreds of thousands of users from different types of applications and performance testing becauseimage
components to be amassed together to provide a complicated production, and unpredictable user loads more and more complex applications, the company put in fear of poor performance occurs, users suffer slow response, system failures and other issues. The outcome was the wastage of income. How to simulate the actual situation? To find a number of computers and the same number of workers to operate in the meantime, and then take a stopwatch to record the reactiin due time? This workshop-style hand-test method impractical, and can not occupy the changes among the program, which will require the auxiliary stress testing tools. The basic strategy is to test the auto load testing, by one or several PC machines to simulate hundreds or thousands of virtual users simultaneously execute business scenarios, to test the application, and record the time of each transaction, in the navel peak data document server, database status. Through repeatable, matter-of-fact test to thoroughly measure the scalability and application performance, identify problems and optimize system performance. Know in advance the affordability of the system to plan for the end-user configuration of the operating environment provides a strong basis. Concurrent performance of the preparatory work before the test test environment: configure the test environment is an important stage in test implementation, test environment for test results or will seriously affect the authenticity and precision. Test environment, including hardware and software environment, the environment, the necessary hardware environment that the test server, client, network connected devices and printers / scanners and other hardware devices posed by the supporting environment; software environment refers to the measured run-time operating system software, database and the environment posed by other applications. Fully ready a test environment has three advantages: a stable, repeatable test environment, to ensure that the correct test results; ensure the implementation of the technology needs of the test; guaranteed correct, repeatable and cozy to understand test results. Testing Tools: concurrency testing is dark carton testing performed on the client side, is generally not used manually, but automated path to use the tool. At present, concurrent performance testing tools mature a lot, the choice is mainly based on testing requirements and cost performance. Well-known tools of concurrent performance testing QALoad, LoadRunner, Benchmark Factory and Webstress and so on. These test tools are automated load testing tool that can be repeated, the real test, the application can be thoroughly measured the scalability and performance throughout the entire development life cycle, along multiple platforms, automated testing missions can be simulated concurrent execution of hundreds of thousands of users to complete critical business application testing. Test data: In the initial test environment need to enter some appropriate test data, the purpose is to nail and validate the data state of the test cases for testing, before the formal start of the test case to debug the test will be when the official start of the test errors to a minimum. In the critical tests of process when the state is necessary for data backup. Create the initial data means that the appropriate data storage down, restore it when needed, the initial data provide a baseline to assess the results of test execution. Formal test execution, test data needed to prepare the business, such as testing concurrent query service, query for the corresponding charts in the database and a considerable amount of data and data should be able to cover all types of business. Simulate the real environment test, and some software, especially mercantile software for the general public, often need to look at the test's performance in the real earth. Test antivirus software, such as scanning speed, the hard disk layout of the proportion of different types of documents should be as close to the real environment, so that out of the test data have actual definition. Concurrency concurrent types of tests and indicators depends on the type of performance testing tool for monitoring concurrent object performance testing to QALoad automated load testing tool as an sample. Software provides for a variety of test targets DB2, DCOM, ODBC, ORACLE, NETLoad, Corba, QARun, SAP, SQLServer, Sybase, Telnet, TUXEDO, UNIFACE, WinSock, WWW, Java Script and other objects of different monitoring, support Windows and UNIX test environment. Remains the most critical test of the monitoring process of the flexible application of the object, such as the current three-tier structure of the widely used mode of operation, concurrent performance testing of middleware as a problem is on the programme, many systems have made use of middleware , select the Java Script monitoring object, hand-written script, you can achieve the purpose of testing. Automated load testing tool for concurrent implementation of performance testing, basic testing programs to follow are: testing requirements and test content, test case development, test environment preparation, test script recording, manuscript and debugging, script distribution, playback performance test configurationImage
and loading strategies, test execution trailing, results analysis and positioning problems, test reports and test appraisal. Concurrent performance testing monitor different targets, the chief indicators of the test is not the same, the chief test indicators, including transaction processing monitor performance indicators and UNIX resources. Among them, the Transaction Processing Performance indicators embody the trading results, the number of transactions per minute, the transaction response time (Min: minimum server response time; Mean: the mean server response time; Max: Maximum server response time; StdDev: transaction processing server response discrimination, the worth the greater the deviation the greater; Median: the median response time; 90%: 90% of the transaction processing server response time), the virtual number of concurrent users. Application Example: test The purpose of performance testing is to simulate multi-user concurrent access Xinhua Multimedia Database, act key search operations, analysis of system performance. Performance test focused on a larger load for the system pressure of the chief concurrent search operations, the concurrent test and labor test, the system uses B / S mode of operation. Concurrent test design a characteristic time of time at the library, English library, image library for single search term, multi-variable search-type search words and mingled concurrent test cases and other search services. Fatigue test circumstance in the number of concurrent users in the library 200, the test cycle of almost 8 hours of a single search word search. Concurrent and fatigue tests during the same time, monitoring indicators, including transaction processing test performance, and UNIX (Linux), Oracle, Apache resources. Test Conclusion: The Xinhua News Agency Intranet chamber test environment and test environment, 100M bandwidths for the concurrent test cases specified, number of concurrent users the system can withstand load pressure of 200, the maximum number of transactions / min to 78.73, running settled, but with the load pressure increases, the system performance has been attenuated. System can withstand 200 concurrent users of nearly 8 hours perennial cycle fatigue stress, and can basically stable operation. Through the system, UNIX (Linux), Oracle, and Apache resources monitoring, system resources to meet the needs of the concurrent and fatigue, and greater use of hardware resources, there is room. When extra than 200 concurrent users, the control to the HTTP 500, connect and timeout errors, and the Web server reported out of memories peccadillo, the system should further cultivate performance, to support a larger number of concurrent users. Suggested that further optimization software system, full use of hardware resources, dilute transaction response time. Fatigue strength and fatigue testing of large file test reliability of the system is used to support the maximum case number of concurrent users, the business proceeded to perform a time, through comprehensive analysis of trade performance indicators and resource monitoring indicators to determine the system processing performance of the maximum workload intensity process. Fatigue test methods can be used to test automation tools, you can also handperformance test image
Workers programming test, a larger proportion of the latter. Under normal circumstances the server to normal and stable response to the request for the maximum number of concurrent users for some time fatigue test data for indicators of trade execution and system resources monitoring data. If the error does not guide to the successful implementation of the test, the timely adjustment of the test indicators, such as reducing the number of users, shorten the test cycle and so on. There is also a case of fatigue testing is the assessment of the current system performance, under normal business with the system based on the number of concurrent users, for a certain time of fatigue testing. Large value of data tests can be divided into two types: fknow next to nothing ofme systems, arsenal, displacement, statistics, interrogate, and other services for large amounts of data independent data test; and pressure performance testing, load performance testing, performance testing a combination of fatigue comprehensive data test solutions. Large volume of data is the key to the test test data preparation tool to prepare test data can depend on. At present, the speed test is required for fussy operations with speed speed manual testing, the test can be continual on the basis of an average, and tools can be measured and the response time indices compared. Applications on the network performance test applications on the network performance test focused on the use of advanced automation technology fall butphisticated network application performance monitoring, network analysis and network application performance prediction. Network Application Performance Analysis Web application performance analysis is accurate exhibit of network bandwidth, postpone, load and TCP port change is how it affects the user's response time. Use of the network application performance analysis tools, such as Application Expert, able to find the bottleneck of applications, we know thatapplications running on the network at each stage of the application of action occurred in the application of the application thread level issue. Can solve a variety of issues: whether the client is running on the database server, needless requests? When the server from the client to approve a interrogate, the application server is unacceptable to cost the time to contact the database server? Application in production before the predicted response time; applications using Application Expert modified performance over the WAN; Application Expert enables you to fast and easily simulation application performance, according to end-user environment in a different network configuration response time, users set their own decided to apply the conditions of operation of the network environment. Web application performance monitoring in the system after the trial run, you need timely and precise understanding of what is occurring on the network; what applications are running, how to run; many PC is accessing the LAN or WAN; which applications occasion system bottlenecks or resource competition, then network application performance monitoring and network resource management system is fussy to normal and stable operation of the. Use of the network application performance monitoring tools, you can achieve a multiplier efficacy, in which we can provide the tools are Network Vantage. More simply, it is effectively used to analyze the performance of critical applications, location is the root of the problem on the client, server, application or network. In most cases, users more cared about what other bandwidth-intensive applications, which users have the largest network vehicle, this tool can also meet the requirements. Network prediction system catching into list the future development of scalable, network vehicle prediction, the network structure of the shock of change on the user's system is very important. According to the plan and provide timely data to predict network performance forecasts. We use Network Prediction of capacity planning tools PREDICTOR to done: set the level of service, completion date network capacity planning, off-line test networks, network failures and capacity restrict analysis, completion of routine diagnosis, prediction network equipment, transport and equipment upgrades to the entire 网络the impact of the network. From the network management software for network topology, traffic monitoring software from the existing access to traffic message (if no such software can be artificially generated traffic data), so you can get the basic structure of the existing network. Based on the basic structure, according to network structure changes, changes in network traffic to produce reports and drafts to illuminate how these changes influence network performance. PREDICTOR provide the following information: Based on the predicted results help users to upgrade their networks to avoid the use of key equipment over the threshold cause the system performance degradation; which need to upgrade network equipment, accordingly reducing network latency and dodge network bottlenecks; Based on the predicted results shirk the necessary network upgrades. Application performance testing on the server application on the server for performance testing, you can use tools to monitor, the system itself can also be used to monitor the command, for instance, can use the Top directive Tuxedo monitor resource usage. The purpose is to achieve the implementation of the test performance of the servertest images
devices, server operating systems, database systems, application server performance in a comprehensive monitoring, testing conviction as shown underneath. UNIX resource monitoring indicators and monitoring indicators depicted in the average load system portrayal of the normal state, the synchronization process in the final 60 seconds the average rate of the number of conflicts per second Ethernet monitoring the conflict to the number of process / thread among processes and threads commute rate per second the number of CPU utilization CPU swap usage (%) disk disk exchange rate commute rate packet error rate receiver receives Ethernet packets per second, the number of errors when the input rate of packets per second, enter the number of Ethernet packets per second interruption rate of CPU Interrupt output parcel error rate of the number of packets sent per second Ethernet packet input rate of the number of errors per second, the number of Ethernet packets output read rate of physical memory in memory pages read per second, write the number of memory pages rate per second of memory pages from physical memory page file writes the number of memory pages in physical memory or erased from the memory pages per second, exchange rate of the number of pages into memory from physical memory pages and sheet peruse out the number of process of exchange rate barter entered into the process of the process of the exchange rate of the number of swap out the number of system CPU utilization of the process of system CPU utilization (%) user user mode CPU utilization CPU occupancy rate (%) disk of each disk block seconds, the number of bytes blocked the purpose of performance testing aims to verify that the software system is able to achieve the user's performance indicators, also found that the existing software system performance bottlenecks, optimization software, and finally played the purpose of optimizing the system. Include the following 1. Assess the ability of the system, test the load and response time are data that can be used to validate the aptitude of the model of the program and help make decisions. 2. Identify disabilities in the system: the controlled load can be added to an extreme level, and damage it, to repair the system bottlenecks or weak districts. 3. System Tuning: Repeat the test run, adjust the system verification activities have been the expected results, thereby improving performance. Detection software in question: the implementation of a long period of testing procedures can lead to memory leaks occur due to the failure of the program reveals the invisible problems or conflicts. 4. Verify the stability (resilience) reliability (stability): the production load in a certain period of time the implementation of the test is to assess the system stability and reliability are the only way to meet the requirements. The type of performance test performance test types include load testing, stress testing, capacity testing. Load test (Load Testing): Load testing is a performance test environment that runs in the overload of data, the program is able to take. Load test to stress that the system can achieve a peak target. Stress Test (Stress Testing): strength test is a performance test, he was especially low in system resources software system under operation. Strength test to punctuate that the system can be stable under high load conditions, namely in extreme cases, system stability. Capacity test (Volume Testing): determine the system can handle the largest number of users simultaneously online. Observed indicators of performance test performance test mainly through a variety of automated testing tools simulate normal and abnormal peak system load conditions to test the performance index. Load testing and stress testing are all performance tests, the two can be in conjunction. By load testing to determine the workload in a variety of system performance, the target is to gradually increase the load test, the system changes the performance indicators. Stress testing is a system by identifying bottlenecks or can not receive performance points, to get the system to provide maximum service level of the test. In practice, we often will be two types of software testing: bs and cs, either general needs performance indicators which content? Bs are generally concerned about the structure of a common program indicators are as usual (simplified): Web server index Index: * Avg Rps: The average absolute requests per second response times = time / seconds; * Avg time to final byte per terstion (mstes): Average operations per second edge of the number of iterations, some people would confuse the two; * Successful Rounds: Successful requests; * Failed Rounds: the failure of the request; * Successful Hits: successful hits; * Failed Hits: failed to click number; * Hits Per Second: Hits per second; * Successful Hits Per Second: the success of hits per second; * Failed Hits Per Second: the failure of clicks per second; * Attempted Connections: try to link a few; CS structure program, As is usually the general background for the database software, so we pay more attention to the test database indexes: * User 0 Connections: the number of user connections, that is, the number of connections the database; * Number of deadlocks: Database paralysis; * Butter Cache hit: Database Cache's hit of way, in practice we will look at case of multi-user tests of memory, CPU, system resources, call situation. These indicators are actually arise out of performance testing in one: competition test. What is the competition testing, software performance testing competitionimage
use of resources (data records, memory, etc.), and other narrated systems to look his tournament for resources capability. We kas long asin the actual test software structure restricts the alternative of testing strategies and tools. How to select performance testing strategy is the actual go we need to know. General software architecture is divided into several types according to: c / s client / Server client / server architecture based on client / server architecture is based on three-tier client / server dispensed architecture b / s based on browser / Web server three-tier architecture is based on three-tier application server middleware l Web server and middleware based on multi-tier architecture Performance Test l each of the different steps in the implementation of the system architecture, developers may choose different implementations, resulting in actual the situation complicated. We can not have a careful explanation of each technique, a method introduced here merely to provide you how to select the test tactics to help examine the performance in different portions of the software, and then analyzes the overall structure of the performance indicators and performance bottlenecks. As the project and the project, the alternative of measurement, appraisal methods are also different. However, there are some common steps to help us to complete a performance test project. Steps are as with 1. Goal-setting and analysis system 2. Select the test method of measurement 3. Learning-related technologies and tools 4. The evolution of appraisal criteria 5. 6 test cases. Run the test case 7. Analysis of test results and analysis system to set goals for each step of a performance test plan will set targets and analysis of system structure. Only explicit goals and explicit understanding of the system will constitute the test range, to know what kind of test to main the technology. Objectives: 1. Identify buyer needs and expectations 2. The actual business needs 3. System Requirements System components System components included here, the averaging of several aspects: the system type, system configuration, the system functions. Understand the ecology of the content is a clear scope of testing to help us, please select the suitable test method for testing. System Type: System class is that we have to differentiate what kind of technology, the prelude, and corresponding technology to do performance testing can be successful. For example: The system species is bs structure, need to have http protocol, java, html and other technologies. Cs or structure may be to know the operating system, winsock, com and so on. Therefore, the screening system is quite important category for us. System components: hardware configuration, operating system settings are performance testing constraints, the general performance testing tools are used to emulate a lot of the actual test user, the system manipulated below overload conditions. Constitute a different system performance test will get different results. System function: system feature refers to the different sub-systems, office file management system in the subsystem, the appointment subsystems, the system can work is a performance test to simulate the correlate, understand these are required. After select the test method amounts a first step, it will have a clear understanding of the system. Next we will converge on software metrics, and pertinent data collection systems. Measure the relevant aspects of: * * the development of relevant processes of norm-setting, roles, responsibilities, * * development of strategies to develop improved standards for comparing the results of learning technologies and tools associated to performance testing is through tools, simulation a large number of user movements, increasing the load on the system. So it is necessary in mandate to have a decisive knowledge of performance testing tool. We all know thatperformance testing tools generally through winsock, http disc user operations and other agreements. The protocol selection is based on the realization of software architecture (network http protocol commonly selected, cs select winsock protocol), the performance of different test tools, scripting languages are different, such as rational automaton in a level c vu script language. Need to behave performance tests to evaluate assorted performance testing tools, testing tools for each performance has its own specifics, and only after evaluation tools to select the line with the existing software architecture, performance testing tools. Determine the test tool, test personnel need to systematize the learning tools and training related to technology. The development of appraisal criteria is to ensure that the purpose of any test software, pre-defined objectives and compliance requirements. Performance testing is not exception. Therefore, we must develop a set of criteria. There are four models are normally performance testing technology can be used to assess: * the linear projection: with plenty of the past, extension or possible future performance of the test image data composed ofradiate plot, continued use of this chart and Contrast the current state of the system. * Model: using queuing methodology formulas and algorithms predict response time, descriptive nature of the workload data and systems associate * imitation: imitation of actual users test your system to use * benchmark: the definition of the initial test and your test as a standard , use it and all subsequent test results were compared to test case design test case design is to understand the basis of business process software. Design principles of test cases provided by the most minimal of test information, the design goal is to test a possible multiple test elements. These test cases must test tool can be achieved, a different test scenarios to test different functions. Because the performance test is different from the usual test, performance test cases as many as possible the design of complex software is possible that the performance bottleneck. Performance testing tool to run test cases to run through test cases. Mainly as the same environment, performance testing are the test results are not accurate, so run these tests, you need to use a different test environment, configured to run on different machines. Analysis of running test after test results, collecting information, for statistical analysis to find performance bottlenecks. By eliminating errors and other elements, so that test results reflect the realistic situation. Structural analysis of different systems have different test methods, bs we will analyze the structure of network bandwidth, traffic on the user response to the impact of the cs structure, we may be more concerned about the overall system configuration of the user operation. Methods for measuring performance for company applications, many of the performance test method, in which some of the ways to appliance it than the other methods difficult. Performance testing to be carried out depends on the type of desired results. For example, reproducibility, benchmarking is the best way. And from the viewpoint of the current user load test system limit, you should use the capacity planning tests. This article will set up and run the performance tests of several methods, and consult the inconsistency between these methods. Without appropriate planning, on the J2EE application performance testing will be a daunting task, and some confusion. Because for any software development process, must collect requirements, understand business needs and design before the actual test during the official timetable. Performance testing requirements driven by business needs, by a team of use cases to specify. These use cases can be based on historical data (eg, server load-week mode) or projected similarity. Clear need to test the content, you need to know how to test. Early in the development stage, you should use benchmarks to determine whether there is an application performance regression. Benchmarks can be in a relatively short period of time to collect repeatable results. The best way to benchmark is to test changes each time one and only change one parameter. For example, if you want to know whether it will increase the JVM memory affect the performance of an application to increment the JVM memory (for instance, from 1024 MB to 1224 MB, then 1524 MB, the last is 2024 MB), the results collected at each stage and environmental data, record information, and then go to the afterward stage. Analysis of test results so that there is a pattern. Next section I will introduce what is benchmarking and best run the benchmark parameters. The late stage of development, the bug in the application have been resolved, the application reaches a steady state, you can run more complex tests to determine the system under different load patterns of performance. These tests are referred to as capacity planning, testing, penetration testing (soak test), peak 测试 (peak-rest test), they are designed to test the application's reliability, robustness and scalability to test close to the real world scene. For the following description should be understood from the abstract sense,88868.org, because each application usage patterns are different. For example, capacity planning tests typically use a slower ramp-up (is defined below), but if the application in a time of day in a rapid explode of traffic, then the test should be modified to reflect the natural situation . But remember, because the change of the test parameters (such as ramp-up period or the user's time to consider (think-time)), the test results must also change. A nice way is to run a array of benchmark tests, establishment of a known controlled environment, and then compared to the changes. The key benchmark is the benchmark to obtain invariable, reproducible results. Reproducible results has two benefits: reducing the number of re-running the test; of the tested products and the resulting number is more sure. Performance test tools used in the test results may have a meaningful impact. Two indicators presume that the test is the server's response time and throughput, they would be the impact of load on the server. Load on the server by two factors: the connection at the same time communicate with the server (or virtual users) the number of requests for each virtual user to consider the width of time between. Obviously, the users communicate with the server the more load will be. Similarly, the request to consider the shorter the time between the load is also greater. Different combinations of these two elements will produce different server load level. Remember, as the server load increases, throughput will continue to rise until it reaches a point. Note that the throughput at a steady rate, at a certain point and then stabilize. At some point, the implementation of the queue began to grow, as all the threads on the server are in use, incoming requests will no longer be dealt with quickly, but placed in queue, re-treatment when the thread is laze. Note that the first time, the implementation of the queue length is naught, then began a steady rate. If so, you should use the flat type of test, all of the users at the same loaded to the server.