Beruflich Dokumente
Kultur Dokumente
-
2020
Performance
Testing
With
Apache JMeter
Executive Summary
As far as testing is concerned, it is an important part of software development. Not only SDLC
is required to build the softwares but STLC also play a big role in the development of quality
product and making customer happy. So, whenever the quality comes, Testing comes into
play. Software Testing is an important activity to control the quality of products/softwares.
There are many types of testing which a software or application pass through during Quality
Control activity. We are not to elaborate each and every type here. We are going to elaborate
only performance and load testing which is a part of performance testing. Performance testing
is about to check the speed, response time, and effectiveness of the software under specific
workload. It helps us to verify the product performance before handing it to client.
So, in this report we have given a detailed description about Performance Testing, How
performance testing is done, and a detailed description about the tool that is ‘’Jmeter’’
which we are going to use for performance testing of our website. Meter can be used as unit
test tool for JDBC database connections, FTP, LDAP, Web services, JMS, HTTP, and
generic TCP connections. JMeter can also be configured as a monitor, although this is
typically considered an ad-hoc solution in lieu of advanced monitoring solutions.
Table of Contents
Executive Summary................................................................................................................................ 3
Overview ................................................................................................................................................ 6
1.1 What is Performance Testing? ..................................................................................................... 6
1.2 Parameters to Check .................................................................................................................... 7
1.2.1 Capacity ................................................................................................................................. 7
1.2.2 Latency .................................................................................................................................. 7
1.2.3 Response Time ...................................................................................................................... 7
1.2.4 Throughput ........................................................................................................................... 8
1.3 Why do we need Performance Testing........................................................................................ 8
1.4 Main Types of Performance Testing ............................................................................................ 8
1.4.1 Load Testing .......................................................................................................................... 8
1.4.2 Stress Testing ........................................................................................................................ 8
2.0 Core Activities of Performance Testing ........................................................................................... 9
2.1 Identify the Test Environment ..................................................................................................... 9
2.2 Identify Performance Criteria .................................................................................................... 10
2.3 Plan and Design Test .................................................................................................................. 10
2.4 Configure the test environment ................................................................................................ 10
2.5 Implement the Test Design ........................................................................................................ 10
2.6 Execute the Test ......................................................................................................................... 10
2.7 Analyze the Result...................................................................................................................... 10
3.0 Test Design ..................................................................................................................................... 10
4.0 Test Environment ........................................................................................................................... 11
4.1 Software ..................................................................................................................................... 11
4.2 Hardware ................................................................................................................................... 11
5.0 Jmeter ............................................................................................................................................ 11
5.1 Some Features ........................................................................................................................... 12
5.2 It Will Help Answer Questions Such as ...................................................................................... 12
5.3 Up and Running with Jmeter ..................................................................................................... 12
5.4 Installing JDK and JRE ................................................................................................................. 12
5.5 Set JAVA_HOME ......................................................................................................................... 14
5.6 Installation of Jmeter ................................................................................................................. 16
6.0 Basic Concepts ............................................................................................................................... 18
6.1 What is Test Plan? ...................................................................................................................... 18
6.1 How to Write a Test Plan? ......................................................................................................... 18
6.1.1 Open Jmeter........................................................................................................................ 18
Overview
This document provides an overview of performance testing which mainly focuses on Web
application performance testing. The different types of tests involved are performance testing
load testing and stress testing. Apache JMeter is a desktop application used to simulate heavy
load on a server, group of servers, network or object to test its strength or to analyze overall
performance under different load types.
In Normal Terms Performance testing is to ensure that the application respond within the
time limit set by the user or not.
1.2.2 Latency
1.2.4 Throughput
The maximum transactions that the application can handle in unit time.
Today majority of modern applications, especially online application, is focused on the mass,
assumes a joint and / or simultaneous operation of multiple users, and in such cases it is
necessary to measure his performance test under applying load. It is a fairly considered one
of the most technically complex types of software testing, because it requires engineers and
testers’ extensive technical knowledge and experience in programming.
Depending on the application architecture, we select the necessary tools to define with the
customer the required level of performance and the threshold number of concurrent users.
Based on these data, we simulate the required number of concurrent users and control the
quality and speed of service under such a load.
Load testing is subjecting your system to statically representative load. Measure the
application's performance under applying user load.
Main Goal is to identify application's performance bottlenecks before goes live. Load
Testing is the process of putting the load through (HTTP, HTTPS ETC) calls on any
Software system to determine its behavior under normal and high load conditions. Load test
Helps identify maximum requests a software system can handle. It helps determine the point
of bottleneck due to which application starts degrading performance. Load testing is
effective when we simulate Real user scenario. We need to know how our users behave on
our application. Try to replicate their behavior using different API calls and generate load
through many concurrent requests at our application for an extended period of time to
understand Application behavior changes.
Stress testing is the process of trying to break the application, which is under test and
identify that breaking point. Subject of the system to extreme pressure in order to verify that
system is getting failed or not. A stress testing is designed to test that how much heavy load
the system or web application can handle.
When we generate load on the system beyond normal usage (high load) to understand
application/system behavior, we call it stress testing. Stress testing is done to know the limits
of an overall system, understand kind of errors thrown during such load and how load the
end user is affected. This helps in planning for events when a much higher load is expected
by provisioning more infrastructure or optimizing applications.
We can use Apache JMeter for load testing static as well as dynamic resources which can
generate concurrent users to simulate real user scenario. It also provides graphical analyses
of test reports. It is written in JAVA, so we need to have java runtime installed to run Jmeter.
Identify the physical test environment and the production environment as well as the tools
and re-sources available to the test team. The physical environment includes hardware,
software and network configurations.
Identify throughput, response time and resource utilization goals and constraints.
Identify key scenarios, determine variability among representative users and how to simulate
the variability.
Prepare the test environment, tools and resources necessary to execute each strategy.
Identify all the scenarios for a specific web application. For example, consider an online test
web application. The key scenarios are
Start Test
Submit Test
Identify the activities involved in each of the scenarios preparing script. Identify the
scenarios that are most commonly executed or most resource intensive. For example, for an
online test web application, the main scenario will be
Login To Application
Click Attempt Test
Start Test
Submit Test
Log Out
Identify work load. The work load model defines how each of the identified scenario is
executed.
Preparing the test environment is the next important step. To run the load test, we need to
identify and set up the machine configuration.
4.1 Software
Operating System
Java Installation
Performance Testing Tool
4.2 Hardware
When we perform a load testing, the number of virtual users a machine can generate depends
on RAM and processor.
5.0 Jmeter
In the previous section, we covered the fundamentals of conducting a performance test. One
of the areas performance testing covers is testing tools. Which testing tool do you use to put
the system and application under load? There are numerous testing tools available to perform
this operation, from free to commercial solutions. However, our focus in this report will be
on Apache JMeter, a free open source, cross platform desktop application from The Apache
Software Foundation. JMeter has been around since 1998 according to historic change logs
on its official site, making it a mature, robust, and reliable testing tool. Cost may also have
played a role in its wide adoption.
Here are some features for which Jmeter is most wanted testing tools among testers.
Performance test of different server types including web (HTTP and HTTPS), SOAP,
database, LDAP, JMS, mail, and native commands or shell scripts
Complete portability across various operating systems
Full multithreading framework allowing concurrent sampling by many threads and
simultaneous sampling of different functions by separate thread groups
GUI (Graphical User Interface)
HTTP proxy recording server
Live view of results as testing is being conducted
JMeter allows multiple concurrent users to be simulated on the application allowing you to
work towards most of the target goals mentioned earlier in the report, such as attaining
baseline, identifying bottlenecks, and so on.
Now let's get up and go running with JMeter, beginning with its installation.
Before Downloading and installing the Jmeter, make it sure you already have
Windows Operating System
Java (JDK and JRE)
To ensure the installation of Java on your machine, check it through Command Prompt by
giving following command.
Java –version
If you don’t have Java installed in your machine, follow the steps given below and get it
installed.
1) Go to http://www.oracle.com/technetwork/java/javase/downloads/ index.html.
2) Download Java JDK compatible with the system you will be using to test.
3) Double-click on the executable and follow the on-screen instructions.
Similarly, you have to download the JRE for your machine to get the Jmeter work on it.
After getting downloaded, JDK and JRE will look like that
To get Java Home Settled on machine we need to follow the step given below.
1) Go to Control Panel.
2) Click on System.
3) Click on Advance System settings.
4) Click the Environment variable
5) Following Window will be Opened
We need to edit the path variable by adding new variable of JDK and JRE. Once we are done
with that click OK and now we are ready to go with Jmeter Installation. Hurrah
We can go for apache JMeter after configuring Java to set environment variable at
JAVA_HOME.
Now extract .zip file to your system as per platform requirement. I am working with
Windows environment so I choose apache-JMeter.zip file, after extract we need a file
JMeter.bat from bin location in JMeter.
After extracting .zip file on our desired location, we will be having a folder with all the files
in it. As mine folder is on Desktop
Just open it and go to Bin directory to get the .bat file run by double clicking on it.
Before going for Performance Testing, we need to go through some very basic concepts in
Jmeter. These must be clear to someone who wants to perform a test in Jmeter.
A Test Plan can be viewed as a container for running tests. It defines what to test and how to
go about it. A complete test plan consists of one or more elements such as thread groups,
logic controllers, sample-generating controllers, listeners, timers, assertions, and
configuration elements. A test plan must have at least one thread group.
This is a plain and blank JMeter window without any additional elements added to it. It
contains a node.
Test Plan node − is where the real test plan is kept.
Elements can be added to a test plan by right-clicking on the Test Plan node and choosing a
new element from the "add" list.
Alternatively, you can load an element from a file and add it by choosing the "merge" or
"open" option.
For example, let us add a Thread Group element to a Test Plan as shown below.
Adding Thread Group
To load an element from file −Right-click on the existing tree element to which you
want to add the loaded element.
Select Merge.
Choose the file where you saved the elements.
JMeter will merge the elements into the tree.
By default, JMeter does not save the element, you need to explicitly save it.
Any element in the Test Plan can be configured using the controls present in JMeter’s right-
hand side frame. These controls allow you to configure the behavior of that particular test
element. For example, the Thread Group can be configured for a number of users, ramp up
periods, etc., as shown below
You can save an entire Test Plan by using either Save or "Save Test Plan As …" from the
File menu. Or we can also use short keys as Ctrl+Shift+S
We can run the Test Plan by clicking Start from the Run menu item. When JMeter starts
running, it shows a small green box at the right-hand end of the section just under the menu
bar. Or short keys are Ctrl+R
The numbers to the left of the green box are the number of active threads / total number of
Threads.
A JMeter Test Plan comprises of test elements discussed below. A Test Plan comprises of at
least one Thread Group. Within each Thread Group, we may place a combination of one or
more of other elements − Sampler, Logic Controller, Configuration Element, Listener, and
Timer. Each Sampler can be preceded by one or more Pre-processor element, followed by
Post-processor element, and/or Assertion element. Let us see each of these elements briefly.
Thread Group elements are the beginning points of your test plan. As the name suggests, the
thread group elements control the number of threads JMeter will use during the test. We can
also control the following via the Thread Group −
Action to be taken after a Sampler error − In case any error occurs during test execution,
you may let the test either −
6.2.2 Controllers
6.2.2.1 Sampler
Samplers allow JMeter to send specific types of requests to a server. They simulate a user
request for a page from the target server. For example, you can add a HTTP Request sampler
if you need to perform a POST, GET, or DELETE on a HTTP service.
Some useful samplers are −
HTTP Request
FTP Request
JDBC Request
Java Request
SOAP/XML Request
RPC Requests
Logic Controllers let you control the order of processing of Samplers in a Thread. Logic
controllers can change the order of a request coming from any of their child elements. Some
examples are − For Each Controller, While Controller, Loop Controller, IF Controller, Run
Time Controller, Interleave Controller, Throughput Controller, and Run Once Controller.
The following screenshot shows a Loop Controller Control Panel
6.2.3 Listeners
Listeners: shows the results of the test execution. They can show results in a different format
Such as a tree, table, graph or log file.
Graph result listeners display the server response times on a Graph. For Example
Till now, we have understood all the basic concepts which will be used during test
performed by Jmeter. So we are going to perform Load Testing of IBIT official website i.e.
(www.ibitpu.edu.pk)
Using Jmeter, we will test the web server performance working at IBIT. We already know
that what Load Testing is all about.
“Load testing is the process that simulates actual user load on any application or website. It
checks how the application behaves during normal and high loads.”
Before putting the load on webserver, we must be know the normal and heavy load that
server is used to serve for. For the time being, as we don’t know exact load values that server
handles. So to get started with load testing, we will make some assumption about number of
concurrent users that access the website or we will use the values a standard website use to
serve.
Before getting down to actual work, let’s consider a scenario for our load testing. We will
test the website according to that scenario, we will create Test Plan and then Test Cases to
perform the test. The website which we have selected for our Load Testing Scenario is
(www.ibitpu.edu.pk).
In normal days, what activities can students perform are visiting the main page, reading new
notifications, watch semester scheme. In normal days, not too much people will visit the
website. So load will be normal.
But in special weeks like during admission days, there would be a large no of students who
will visit the website and will apply for the admission. So there will be a high load in those
days. We will test our website for both normal and high load.
First step of testing is identifying the key scenarios. We will take scenarios based on their
business value (how important they are for our success) and user preference (where users
spend most of their time). In the test, we will examine the response time for the website's
different pages. The response time is important because loading time is a major contributing
factor to page abandonment. With the increase of the internet connection speed, the average
user patience is going down. If a website's load time is more than 4 seconds approximately
25% of the users will leave. The pages that we have chosen are:
For Normal Load:
1) Home Page
2) Course scheme page
3) Notifications
4) contact page
5) Feedback page
For Heavy Load:
1) Home page
2) Aggregate calculation page
3) Online Application Submission page
1) Go to home page
2) Click on notification and cross it
3) Go to contact page
4) Enter your credentials and submit
1) Go to home page
2) Go to Aggregate calculation page
3) Enter your marks and calculate result
4) Go to on line application submission page
5) Enter credentials and submit application
For our test following assumption will be used throughout the testing.
Normal Load = 30 users
Heavy Load = 100 users
We have assumed that this is the user load for which website have been designed to handle.
So let’s start now with test plan.
After starting the Jmeter, you should see the Graphical User Interface of Jmeter with Test Plan
Node just as shown in figure below.
A test plan is composed of a sequence of test components that determine how the load will
be simulated. We will explain the how some of these components can be used as we add
them into our test plan.
Set the value for three important properties that influence the Load Test.
Set for the Normal load first according to our assumptions.
No of Threads ( Users) = 30 for normal and 100 for heavy load
Ramp-Up Time = 10 same for both
Loop Count = 1 same for both
The HTTP Request Defaults Config Element is used to set default values for HTTP Requests
in our test plan. This is particularly useful if we want to send multiple HTTP request to the
same server as part of our test. Now let’s add HTTP Request Defaults to Thread Group.
Now, in setting set the server name = www.ibitpu.edu.pk Setting the server name here will
make it default for all the items in Thread Group.
Now, we are almost done with the Test Plan except one item i.e. Listener.
Listener is used to capture the result generated by Jmeter. So that we could judge the output
and assess the performance of our server under our defined load. There are many types of
listener which can be used for capturing the results. We can use either one or all the listeners
at a time. No restriction is there for usage of listener.
Set the Port No in Global Setting of HTTP Proxy Server. Always remember that same port
no will be entered while setting web browser proxy. We can choose any one. As in figure
8080
Now, everything has been done perfect in Jmeter. All we need to do now is setting the proxy
of our web browser.
Go to Browser
Then Setting
Search Proxy
Click on open computer’s proxy setting
Set Server name and Port No. same as in Jmeter
Go Setting of Browser
Search Certificate
Click on Manage Certificate
Go to Authorities
Click on Import
Select JmeterRootCA certificate from Bin of Jmeter
Now, we are done with Setting HTTP Proxy. Now we can record over script.
Go to Jmeter and click on Start Button. It will ask you to accept a certificate. Just click on ok
and go to web browser to perform user activity.
Launch www.ibitpu,edu.pk in Web Browser and perform activities whatever you want. Once
you are done with it. Go to JMeter and click on stop.
Every activity will have been recorded in the form of script in target controller.
Now save your Test plan and Run it
Just click on Green Start Button and let Jmeter do its job.
Now, we will record the script for heavy load and will run the test plan for both loads and
will determine whether our website handles that load or not and if handles then at what stage
it start degrading performance.
7.2.13 Results
Now let’s see our results for both normal load and heavy load.
Stats:
Total Samples = 4110
Throughput = 7743.029/minute
Avg. Response Time = 198 milliseconds
These figure shows that website can handle its normal load easily. It has high throughput and
minimum response time to facilitate the visitors as quick as possible.
Now let’s see what are the stats for heavy load, around 100 students will be doing activities
like merit calculation, online form submission etc. during admission days.
Stats:
Total Samples = 3500
Throughput = 3780.786/minute
Avg. Response Time = 1435 milliseconds =1.4 sec
These figure shows that website can handle its heavy load easily. It has high throughput and
response time is 1.4 sec that is not bad.
Stats:
Total Samples = 4532
Throughput = 1040/minute
Avg. Response Time = 7369 milliseconds =7.3 sec
These figure shows that website cannot handle the load of 200 users. If we want our website
facilitate 200 users we have to get our resources optimized. 100-150 users is the maximum
load it can handle scarcely.
When we generate load on the system beyond normal usage (high load) to understand
application/system behavior, we call it stress testing. Stress testing is done to know the limits
of an overall system, understand kind of errors thrown during such load and how load the
end user is affected. This helps in planning for events when a much higher load is expected
by provisioning more infrastructure or optimizing applications.
So here is a Roadmap for Stress Testing Script Recording which we will follow.
Basic function of this website for our scenario is displaying the results.
1) Go to Home page
2) Go to Roll no entry page
3) Wait for the result page
Now let’s run our script with huge number of users and check performance under stress.
8.6 Results:
Users = 2000
Stats:
Total Samples = 23882
Throughput = 4109/minute
Avg. Response Time = 15105 milliseconds =7.3 sec
Max. Response Time = 238447 milliseconds =238.447 sec = 4 minutes
These figure shows that website cannot handle the load of 2000 users. If we want our
website facilitate 2000 users we have to get our resources optimized.
---------------------------------------------------------
The End