Comparison between QALoad, WebLOAD, OpenSTA and LoadRunner

Documented By: - Sravan Tadakamalla

Comparison Between QALoad, WebLOAD, OpenSTA and LoadRunner

LoadRunner QALoad WebLOAD OpenSTA

Virtual User Generator Script Development WebLOAD IDE Modeler

Workbench, Player

Controller Conductor

Analysis Analyze WebLOAD Commander

Console

SCRIPT GENERATORS:

Virtual User Generator (VuGen)(LoadRunner):

VuGen is used to record the flow of the application called scripts. The entire recording part is done in
the VuGen including any enhancements in the scripts (parameterization-including the user defined
variables which contains the values which will be substituted at the RunTime, correlation-handle the
dynamic values, content checks, & sleep time inclusion).
The recording can be done in many protocols (e.g. HTTP/HTML for Web application). LoadRunner
supports nearly 44 different protocols.

Before recording, we need to set the recording options as per our requirements. Recording options can
be available from the start record wizard or from the Tools menu.

After recording, the script consists of three parts, viz. vuser_init, Action, and vuser_end. The
vuser_init contains that part of script which is the starting of the script (e.g. login into any eCommerce
site), Action contains that part which the user actually performs his operation inside the website (e.g.
going through the shopping site and buying different products), vuser_end contains the part when user
is leaving the site. The reason why the script is divided is, whenever we want the script to Run for no.
of iterations only the Action part is to be executed repeatedly.

After completing all the enhancements for the script, we do a sample Run in the VuGen. Before
running we will set the Run-Time Settings for the script. Run-Time Settings can be available from the
Tools Menu option. These Run-Time Settings includes the options like setting the script to execute for
no. of iterations and the time to wait between each iteration, amount of think time to include in the
script, log options, browser to be used, proxy if any and so on.

Parameterization is done by selecting and right-clicking the string to be parameterized and going
through the wizard by giving the values to be substituted and a variable which contain them.
Comparison between QALoad, WebLOAD, OpenSTA and LoadRunner

Correlation is done basically in three ways in LoadRunner. They are

1. Autocorrelation (Ctrl+F8)
2. Manual Correlation(using web_reg_save( ) )
3. Correlation Studio
In correlation studio we create some rules, by giving the left boundary and right boundary, and these
rules are automatically checked and correlate the values while we record a new script. This happens
when we include them through the Record Options.

Script Development WorkBench (QALoad):

The recording part is done in the script development workbench same as VuGen of LoadRunner. Here
we will get the script and that is to be enhanced and Run in the Player. Player is a different component
from WorkBench. We need to upload the script and Run in the Player.

 Here, the Recording Options can be found in the Session Menu -> Record.. option.
 Runtime Settings are seen as Playback Options and these are seen with the vistree(Visual
Navigator Tree). Parameterization can also be done using this view's menu items. The
parameter values can be taken from the Datapool Files. Parameterization is also called
Variablization (in previous versions).
 Correlation is done by using the Extract String wizard or manually. We have Parameterization
Rules which acts similar to Correlation Studio in LoadRunner.
 Here, vistree view can be considered as tree view of LoadRunner.
 In workbench, to get the script we need to right click on some part of the vistree and click on
the view script. The tool does not give us the editable script and to get this, we need to save the
current script with some other name either by using save as option from File Menu or right
clicking on the vistree and selecting Create Editable Script option.
 As we have three sections of the script named vuser_init, Action, and vuser_end in
LoadRunner we have Transaction setup,Transaction Loop, Transaction cleanup in QALoad
workbench.
• In workbench, while recording we have checkpoints in place of transactions.

The recorded script is first available as a capture file with .cap as extension, which then to be converted
to vistree. We have two different conversion options that we can decide after recording the script( here
capturing the script), they are HTML & HTTP forms. HTTP is same as URL based recording in
LoadRunner.
In LoadRunner’s VuGen, we can Run the script for number of iterations by single user, where as in
QALoad’s WorkBench, we can Run the script for only one iteration and this test-run is called validating
the script. To test-run the script for no. of iterations, we make use of Player.

In LoadRunner we record and test-run the script in the VuGen itself but in QALoad to test-run the
script Player is used. Here in Player we validate the script to check whether it is ready to be Run in the
Conductor for multiple iterations.

WebLOAD IDE(WebLOAD):

{Issues:
• There was some problem with the correlation of the dynamic values. This problem is due to the trial
version of the WebLOAD tool. Where as in licensed version there will be a separate option when
Comparison between QALoad, WebLOAD, OpenSTA and LoadRunner

we right click on the code in the DOM view.

• Also the report creation is not so user friendly. There are a lot of statistics to be considered. Here
there is no separate tool for analyzing in the trial version. }

In WebLOAD, recording is done in IDE same as VuGen. Here we will get the script and we have two
types of views, Agenda View & JavaScript View. Agenda View is same as Tree View and JavaScript
View is same as Script View of LoadRunner.
• Unlike LoadRunner and QALoad, here in WebLOAD, we do not have parts in the script.
• One demerit of WebLOAD is we can not include transaction names at the time of recording;
instead we have to manually drag and drop the Begin Transaction, End Transaction icons after
the particular URL which are available in the Load Tab of the Toolbox available in left pane.
We also have timers called Set Timer, Send Timer which acts as transaction start and end resp.
and also sends the time to the Console while Load test.
• We can Run the script in the IDE to check whether the script is ready for the Load test.
• We call compilation process as Check Syntax in IDE.
• Parameterization is done by dragging and dropping the Global Input File in the Agenda Tree
from the General Tab of the Toolbox in the left pane.
• Correlation is done by catching the hidden fields in the DOM view and extracting that values
and assigning to a variable.

Modeler(OpenSTA):

Modeler works the same way as VuGen. Modeler is used for recording the scripts.

TEST EXECUTION ENVIRONMENTS:

Controller(LoadRunner):

We create the Load scenario and Run it in the Controller. The scenario is the predefined method for the
test to Run. This scenario can include one or more number of scripts.
The scenario which includes more than one script is called Group Scenario. When we Run the group
scenario, all the scripts are being Run simultaneously with assigned no. of users.

The scenarios can be of two types based on the functionalities: Manual Scenario & Goal-Oriented
Scenario.

Goal-Oriented Scenario: In the Goal-Oriented scenario, users can set the pre-requisites of the test like
maximum number of hits to be achieved, throughput, etc. These can be called as goals of the test to be
achieved.
Manual Scenario: In the manual scenario, users don’t have the choice of setting the goals. The scenario
will Run normally.
In Manual Scenario, we have percentile mode option.
Here we also have scheduled tests through which we can Run the test at a later time.

LoadGenerator: The load (no. of vusers) can be distributed among other systems using the
LoadGenerator. LoadGenerator can assign the load to different machines, and simulate a different
group of users performing different activities on the application being tested at the same time. And this
Comparison between QALoad, WebLOAD, OpenSTA and LoadRunner

‘different group of users’ belongs to different machines.

Reporting: The LoadRunner’s Controller is capable of generating graphs which show different metrics
like transactions per second, hits per second, etc. separately. LoadRunner sends the statistics of the test
to the Analysis in the form of reports.

Debugging: We can see the flow of the scripts running when we select the ‘view test log’ by right-
clicking on the script name.

We can monitor the sever performance by using the Monitors functionality in the LoadRunner’s
Controller. Monitors allow us to mention the server’s resources to be monitored while test is running.
At the end of the Load Test, we will get the statistics of the server resources through monitors. We can
also select Databases to monitor.

Conductor(QALoad):

The Conductor of QALoad is used to Run the Load Test. This serves us similar to the Controller of
LoadRunner.

The term we use in conductor is session in place of scenario of controller.

We don’t find the scenario types based on the functionalities here, as in LoadRunner. QALoad has
‘batch test’ which is the collection of one or more sessions. We can add one or more no. of scripts to the
single test session. The batch test will execute the scripts in it linearly one after the other and not
simultaneously as in LoadRunner.

We Run the Load test in the Conductor by configuring the machine by giving the machine name or IP
address manually.
In conductor we can assign the VUs (vusers) to each machine. We can configure and add different
machines for test using the configure machines wizard. This is same as what LoadGenerator does in the
LoadRunner.

Here in conductor instead of ramp up and ramp down time we will specify the time interval, starting
VU’s and ending VU’s. It automatically takes the time.

The reports generated for the test in the Console are sent to the Analyze module and we can also view
the graphs in the Console while test is running. And as soon as we exit out of the test, the Analyze
module will open showing the graphs and reports.

We have Monitors functionality in QALoad’s Conductor to monitor the server performance while the
Load Test is running. Apart from this functionality, QALoad also comes up with some additional
tools/modules like Application Vantage and Server Vantage. These modules give the advantage for the
Testers in running Load Tests and for Performance Tuning issues.

Console(WebLOAD):

In Console, we call the scenario of test as Agenda.

An Agenda is a test-script used to test our application. WebLOAD supports:
Comparison between QALoad, WebLOAD, OpenSTA and LoadRunner

• Single Agendas- scenarios which contain single test script.

• Mix of Agendas- a combination of Agendas, each simulating a different group of users
performing different activities on the application being tested at the same time.

We don’t find the scenarios’ types based on the functionalities here as in LoadRunner.

Console is the module which acts as both Controller and Analysis modules of LoadRunner. In Console,
we can Run the test and simultaneously we can generate the reports.
We also have different tools like Performance Measurement Manager (PMM) and Report Manager
which are additional tools used for monitoring the server and better analysis of the test. (These are
available for licensed versions)
There is an advantage with this tool which allows us to save the Load Session by which we can open in
a later time and add reports for the load session.

We use the terms template and load session in Console, where template is the settings applied to Run
the load session and load session is the resultant test. That is, template is similar to scenario in
LoadRunner, when we save the template the scripts allocated and the load balancing between them is
saved. Whereas load session is like the resultant report we get in the Analysis after the test using which
we analyze the test.

We can export the reports to Microsoft Excel sheet and also as a HTML document.

REPORT ANALYZING ENVIRONMENT:

Analysis:

The Analysis module of the LoadRunner is used for analyzing the test we ran in the Controller. We can
analyze the test by using the different graphs available and the reported values for different metrics.
Using the Analysis module, we can create the custom reports using different metrics we consider for
the particular application.

We can export the reports to Microsoft Word document and also as a HTML document.

Analyze:

The Analyze module works in the similar way as LoadRunner’s Analysis module.
We can export the reports to Microsoft Word document and also as a HTML document.