Welcome to AdvancedQTP’s review of Quicktest Professional 10 (QTP Atlantis).

QTP 10 has recently come out (early February), and is now available for evaluation from HP’s
main site (you can find it here). QTP 10, like QTP 9.5, is also delivered in a complete DVD that
contains all the add-ins and SDK packages, so brace yourself for a lengthy download (1.3GB
zipped, 1.8GB extracted).

How this review is structured: The review is built as a free-style walk-through across QTP. If you
are interested in a concise feature breakdown, scroll down to the end of the review for a
feature-by-feature summary.

Disclaimer: Many of the killer feature of QTP 10 only work when it’s coupled with Quality Center
10. As I don’t have a working Quality Center just sitting there, waiting to be upgraded, I was
unable to cover those features. You can read more about my impressions of them in our QTP 10
live demonstration breakdown. I will just say that they are very impressive, and bring real added
value to the QTP-QC dual.

The Install Process

Installing QTP 10 is pretty straightforward and is practically identical to installing QTP 9.5.
However, unlike QTP 9.5, you can upgrade your previous installation, instead of removing it
completely (Personally, I think removing the old one is for the best – no pesky leftovers to
contaminate your system). If you have any add-ins installed, they would also be automatically
upgraded.

Page 1
© All rights reserved – Solmar Knowledge Networks
 Figure 1 - The Main Install Screen

Figure 2 - The Upgrade Dialog

Page 2
© All rights reserved – Solmar Knowledge Networks
 Figure 3 - Add-Ins Selection Page

I ran the upgrade from a wireless network drive, through my MacBook Pro, to an XP virtual
machine, and it only took approximately 10 minutes. I guess this is a much elaborated way for
saying the install process is very fast.

After the installation is complete, you can expect to find all the familiar icons at their familiar
places, with one significant addition : an out of the box installation of extensibility help files for
.Net, Web, Java and Delphi. I shall address the matter of extensibility later on.

Figure 4 - Extensibility Help

Page 3
© All rights reserved – Solmar Knowledge Networks
Overall Look and Feel
QTP 9.0+ users will feel right at home. Most of the user interface is unchanged. At first glance,
the IDE seems to be completely identical, from the default panes to the start page (the start-
page help links have been updated, of course).

The menus are now presented as trees rather than tabs, giving a more updated look. This is
undermined, though, by the old-looking ridged buttons and panels, creating an unpleasant
overall feeling. It is immediately noticeable when compared to the option tree in Visual Studio,
which is more UI consistent and soft. But maybe it is just that I am too demanding.

Figure 5 - QTP option tree as compared with VS

(on the left – QTP: ridged and unpleasant; on the right – VS: soft and consistent).

The open / save dialogs were also updated to display a much friendlier big-button style:

Figure 6 - Save / Open Dialog

Page 4
© All rights reserved – Solmar Knowledge Networks
Even with all the extra text-parsing features, QTP seems to run more smoothly (which quite
surprised me), and opens / saves large tests slightly faster than QTP 9.5; at least for me. The
overall feeling is very responsive, and as an extra bonus, you can define a global Stop-Test
hotkey in case QTP hangs on an endless loop, and the window is inaccessible.

Figure 7 – The (same old) IDE

Custom Toolbars

A nice new feature presents the ability to customize the toolbars and button layouts. Any menu
command can be planted in any of the toolbars (though you cannot add new toolbars of your
own). The process is pretty intuitive, and basically only requires you to locate the relevant
command (the All Commands category helps in this regard), and drag it to the toolbar you want
it in.

Page 5
© All rights reserved – Solmar Knowledge Networks
Figure 8 - Selecting a command to add

Another nice touch is the ability to add external commands to the tools menu – Just select an
external tool / file, and it will be available though the tools menu.

Figure 9 - Adding an external command (left), and how it shows in the tools menu (right)

Version Control and “Save to go”

The file menu already hints some of the new killer-features of QTP 10: Version control (only
works with QC), and saving a test with all its resource (which I like to think of as “Save to go”).

Page 6
© All rights reserved – Solmar Knowledge Networks
 Figure 10 - Source control, and "Save to-go"

As I could not work with the version control features, I will only refer you to our QTP 10 live
demonstration breakdown, where I reviewed them, and was much excited by it. However, I was
able to play with the “Save To-Go” feature, and I must say it’s just what the doctor ordered.

We were all frustrated at one point or another, by the impossible task of transporting our test to
a different directory / computer. It seems that no matter how much we tried to keep all our
resources organized, one of them would always slip through, leaving us with a broken tests and
a weary heart. Well, No more!

Now we can simply choose “File->Save test with resource” which will display the test’s resource
tree; you can also choose either to save it to a new location, or to export it and all its resources
to a compressed ZIP file (smart – shows HP has kept the user, not only the feature, in mind).
Once you hit the Save button, QTP will save the test to the location you specified, copying all the
function libraries, object repositories and recovery scenarios to dedicated folders, and re-linking
them to the main test (via their relative paths). In addition, you’ll be presented with an HTML
summary of the procedure.

Figure 11 - "Save to-go"

Page 7
© All rights reserved – Solmar Knowledge Networks
I messed a bit with the entire mechanism, and I am happy to say it works perfectly. It is a
straightforward solution to a very nagging problem. Well done!

Tasks and Comments

Another area that may appear unchanged at first glance is the IDE panes. You still have all the
old ones (DataTable, Missing Resources, Debug, Active Screen, Available Keywords, Resources,
and Test Flow), and hiding among them is the new To-Do pane.

The To-Do pane contains two tabs – Tasks and Comments. Tasks are a structured way to create
and manage ongoing project tasks, and comments do pretty much the same thing in a more ad-
hoc, on-the-fly manner.

Whenever you write a comment starting with “To Do”, “todo”, “to-do”, or “TODO”, the editor
will automatically pick that up and show it in the TODO comment box. Double clicking on the
comment will jump to the relevant position in the code, and usually, that’s pretty much all you
need – a concise reminder of the issue at hand, and an anchor to the relevant position.

Figure 12 - The To-Do Pane - Comment Tab

The comment parser usually only operates on the test’s internal actions, but you can make it
also parse the external actions and open function libraries that are associated with the test. I
guess it’s not the default behavior due to performance issues, although even in extremely large
tests, the parser didn’t put much of a strain on my system – only a split second overhead
(impressive, considering the fact it runs on every cursor move, and not just on file save).

While the comment feature was perfectly implemented, and is very useful indeed, the Tasks
feature left me puzzled and unsatisfied. When adding / editing a task, you can specify certain
details such as its subject, priority etc. Personally, I don’t see any added value here, compared to
writing that meta-data as a simple TODO comment.

Page 8
© All rights reserved – Solmar Knowledge Networks
 Figure 13 - Task details

If I ever need added control and manageability for my tasks, I’d use a dedicated task application
(such as the tasks in outlook, etc.); otherwise, I would much rather just quickly write a TODO
comment, and be done with it. The fact that I cannot link a task to a specific code-line creates
the absurd situation where ad-hoc comments are more powerful (in this sense) than the
structured tasks.

To sum up, I liked the Comment feature, and once I could use that, I see no point to the Tasks
feature. If you’d like, you can export both the TODO comments and the tasks to an XML / XLS /
CSV file.

Report Improvements
Well, comments and tasks might help us to organize our workflow, but don’t really extend QTP’s
automation features and possibilities. To begin our exploration of features that are directly
linked to automation, let’s take a closer look at the results and the reporter.

The most significant new feature to the results and reporter is the Local System Monitor. You
have the ability to monitor several resources on your computer throughout the run-session, and
define custom thresholds for them. Values that exceed these thresholds will fail the test.
Defining the counter is done via the File->Settings menu, and it’s a quick and easy process. There
are many different counters, and each one is accompanied by the relevant explanation, so you
won’t have to guess what “Virtual Bytes” (for example) is.

Page 9
© All rights reserved – Solmar Knowledge Networks
 Figure 14 - Defining system monitors

The system monitor feature is fused seamlessly with the report – it has a dedicated tab next to
the Screen Recorder, and each result node you select will focus on the relevant spot on the
resource graph. Similarly, if you select a point on the graph, the closest result node will be
automatically selected in the test steps tree.

The graph itself allows us to zoom, hide / display specific monitors, and clearly displays the
monitor limit we’ve defined (if any). The useful UI, combined with the Step-Graph Sync makes
this feature extremely powerful and fun to use (yes, work can also be fun). I expect this ability to
develop into a sort of an impoverished load test, although the ability to monitor only one
process may be too limiting in that regard.

You can also access the monitors throughout the test-run via the SystemMonitor.GetValue
command, allowing you to write your own custom reports, decision structures and checkpoints
based on the monitors data.

Page 10
© All rights reserved – Solmar Knowledge Networks
 Figure 15-The graph and the steps tree are in perfect Sync

Not only do the System Monitor Graph and the Screen Recorder sync with the step tree, but the
results now offer the ultimate step sync – right click any node in the test-step tree, and select
“Jump to step in QuickTest”. QTP will focus on the relevant script line within the relevant action.
I was really excited about this feature, until I realized that if you’re using a custom report
function (and let’s be honest – we all do), QTP will always jump to that one function, making the
whole thing completely useless.

Another long-awaited feature is the ability to export the results to more formats. You can export
the Screen Recorder movie (as before), the system monitor data, or the report itself – either to
HTML, Doc or PDF. The export works very well, and it’s quite customizable – you can still choose
a short or long format, or point to your own custom XLS file, which will define the final data,
look and feel of your report. The only thing missing is an API call to do this “from behind the
scenes”, or automatically at the end of each run session.

Page 11
© All rights reserved – Solmar Knowledge Networks
 Figure 16 - The Export Report window

One last new feature to the reporter is the out-of-the-box ability to attach images to report
nodes. Forget about all your elaborate workarounds for attaching images to the report – in QTP
10 just need to add the picture file path to the report command. I’ve tried it out, and it works
well enough.

Intellisense and Auto-complete

Besides the QC-integration features, the biggest and most significant change to QTP has been in
QTP’s internal intellisense and auto-complete mechanisms. For the first time, we have both
variable auto-complete, as well as COM intellisense up to any hierarchy and level we’d wanted.
Needless to say, these features got me very excited, and that may have been part of the reason I
was so disappointed by them.

Don’t get me wrong – things are much better with these features than without them, but when I
think of what they could’ve been, I can’t help but feel that a great deal of potential remains
untapped.

But first, let’s acknowledge what does work fine – even great – about these features. The
variable auto-complete works just like the functions auto-complete – just type in the first letter
or letters of a variable, hit Control-Space, and QTP will auto complete it, or at least present a list
of viable options. It works both on your current action, as well as on every associated function
library.

Page 12
© All rights reserved – Solmar Knowledge Networks
Figure 17 - Variable auto complete

On a basic level, the COM auto complete feature also works fine – Insert any object into a
variable, and you will have intellisense for all its inner methods and properties, up to as many
level s as you’d like. The process is immediate and transparent to the user. Any other variable
that will receive this reference will present the same intellisense, as will any function that
returns it.

Figure 18 - Intellisense on an Excel object. Notice the auto complete list is shown for a child property

Figure 19 - Intellisense is passed between functions and variables

The intellisense mechanism works on more than script created COM objects. You can use it on
test objects in your AUT. Just add the relevant object to the object-repository (will not work with
DP), use the . Object property to punch through to the AUT’s runtime environment, and

Page 13
© All rights reserved – Solmar Knowledge Networks
navigate on through the intellisense. This is really wonderful in rich environments such as .Net,
Java and Web, where the objects have an unending variety of runtime properties, function and
inner objects.

Figure 20 -Inserting a Test-Object into a variable, and using the inner intellisense to spy on nested properties and
methods

This seemed to me like the ultimate solution to the Object-Spy inability to drill down through
complex runtime properties. And indeed it’s a valuable exploratory tool, and it greatly improves
the nonexistent intellisense in prior QTP releases.

A very nice touch is the inclusion of Tarun Lalwani's Intellisense Resizer in QTP 10. You can now
drag the edge of the auto complete window to widen / narrow the display, which comes in
handy when you’re dealing with long property / method names.

The Debug pane has also undergone some improvements, though these are more UI related
than meaningful advancements. You can now see the type of the variable you’re monitoring,
and familiar objects now have a little plus sign for expanding object properties.

Figure 21- The new debug pane look. Notice the variable type column

So, if everything is so great, why am I so disappointed? Well, critical little bugs and overlooks still
prevent this from becoming a significant enhancement for advanced programming in QTP.

So now, let’s turn to the bad stuff:

Page 14
© All rights reserved – Solmar Knowledge Networks
First of all, neither the auto complete nor the intellisense work for VB Script classes. As one who
uses them extensively throughout his code, this deeply saddened me (even though I knew
beforehand this would be that case)

As for the auto complete – Variables are not recognized when declared with the Public or
Private keywords (only with Dim statements). Also, variables defined with brackets (e.g. [Dim])
show up on the list of possibilities, but are never automatically completed. And while these are
mere annoyances, the following present real difficulties.

There is no intellisense for functions defined in associated library files, only for those defined
within the actions themselves (Variables directly defined in associated libraries are parsed, but
not functions). I’ll say it again – There is not intellisense for functions stored in associated library
files. Personally – I think this is a no-go issue. But in any case, the function’s result is parsed at
the lowest possible code-line, so for example, in this function:

Figure 22 - The intellisense parser is blind to decision trees

The lowest return value is nothing (appears in line 6), so the function will always have the
intellisense of Nothing. In this case you can just switch the two cases, but what of a function that
returns 3 possible values? Or have multiple exit gates?

Actually, things are even worse – functions inner variables leak out and contaminate the entire
action scope. Objects put into an inner function variables will show up on the intellisense of
outer variables with the same name, as shown here:

Page 15
© All rights reserved – Solmar Knowledge Networks
Figure 23 - Notice how the global variable is affected by changes made to the function's inner variable

But, at least we have the Test-Object intellisense, right? Well, things aren’t perfect in this area
either. It seems that the intellisense chain is broken whenever a collection is involved. For
example, no further intellisense is provided for this line, since we’ve passed through a
collection:

Since many of the interesting properties in .Net, Web and Java are stored in collections, this
presents a real problem. It’s not that this feature is entirely without merit, but just a few final
touches from being amazing. Update: I’m happy to say that HP also acknowledges the
importance of this issue, and they plan a patch that will allow intellisense to work though
collections. It’s supposed to be released shortly, and we will update you when it does.

Another related major issue is breaking the PDM.DLL debug hack, which allowed for some sort
of limited intellisense in the debug pane. In QTP 10, registering PDM.DLL version 9+ will cause
the debug pane not to show any object variables whatsoever – not in the Watch tab, nor in the
Variables tab. This was quite a crippling blow to me personally, as I’ve grown to relay on it,
though I guess it’s not fair to cry over a discontinued backdoor hack that was unsupported to
begin with.

So, in conclusion, these are all worthwhile features, but they are all deeply flawed. While they’ll
make your life easier, they do not bring the revolution I expected of them.

Another “almost” feature is highlighted code regions. Unlike most modern editors, which
include folding code regions (i.e. regions you can collapse and expand using + and 0), QTP 10
only highlights the current code block with a blue box. While this provides some visual aid, it’s
severely outdated for a 2009 product (or a 2000 product for that matter).

Page 16
© All rights reserved – Solmar Knowledge Networks
Figure 24 - Highlighted code regions (QTP - left), compared to collapsible code regions (VS - right)

Improvements to Actions and Checkpoints

QTP 10 includes several improvements to the actions and checkpoints mechanisms. The most
significant improvement is the new ability to dynamically call an action, without inserting it to
the test beforehand. This can truly revolutionize the interoperability of cross project tests and
actions, and I’m very happy this feature found its way into the version.

All the run-time debug abilities will be available for these dynamically added actions, so you’re
not giving anything up by using this feature. Just add this command in your script, and it will
execute when the test run:

LoadAndRunAction(TestPath, ActionName, [Iteration], [Parameters])

As for checkpoints – Bitmap checkpoints has been given quite a buff – in addition to the ability
to view the difference between the expected and actual results, you can now build your own
custom comparison DLL, and integrate it with the native checkpoint mechanism of QTP. There
are lengthy reference documents and tutorials explaining exactly how this can be done.

In addition, QTP’s automation API has been updated, and among its many changes, those
dealing with actions really stand out. You can now read an action’s script, edit it or overwrite it
entirely before starting the run session. You can also add a new action to the beginning or the
ending of the current test. There is plenty of potential in these new API interfaces, and I can’t
wait to find new and exciting uses for them (parse a script to automatically add class
inheritance?).

Environment and Extensibility Improvements

Beside the constant expansion of the supported environments and applications, QTP 10 also
includes several improvements to its extensibility packs.

Web extensibility now includes an out-of-the-box limited support for standard AJAX controls; a
global Window object to be used within JavaScript to access the IE namespace; Extensibility
code can now be debugged by the Script debugger or Visual Studio debugger; and there are
minor improvements to reporting capabilities.

Page 17
© All rights reserved – Solmar Knowledge Networks
.Net Add-In and Extensibility now includes VS2008 project templates; Attaching to property
grids (via the new SwfPropertyGrid object); Added the TO property and method for reading
tooltip error text (via ErrorProviderText) ; and support for custom table checkpoints;.

Terminal Emulator Add-In now has a Validate option in the configuration page, allowing you to
validate your emulator configuration before the actual run session.

Delphi is now also supported, with a SDK for building your own custom Delphi extensibility.

The following lists the new environments that QTP 10 now supports:

Windows 2008 Server 32-bit Edition

Windows 2008 Server 64-bit Edition
Windows Vista, Service Pack 1, 32-bit Edition
Windows Vista, Service Pack 1, 64-bit Edition
Windows XP Professional 32-bit Edition—Service Pack 3
Citrix Server 4.5
Internet Explorer 8, Beta 2
Firefox 3.0.x
Delphi: IDE, versions 6, 7, and 2007 (for controls based on the Win32 VCL library)
SAP: CRM 2007 (For controls that support test mode enhancements. Requires SAP
notes: 1147166, 1066565, and 1002944. Later SAP notes related to test mode
enhancements are not supported.)
Java: IBM 32-bit JDK 1.5.x, SWT toolkit version 3.4
Java Extensibility: Eclipse IDE 3.4
.NET: .NET Framework 3.5—Service Pack 1

Text Recognition has also been given an upgrade, and now you can select specific languages to
be processed, and also the OCR mode (single / multiple block).

Page 18
© All rights reserved – Solmar Knowledge Networks
Figure 25 - The new text recognition option screen

Summary and Conclusion

QTP 10 brings many improvements to the user’s central workflows, and adds some real value
with these. The two main killer-features show great potential: The QC integration features
(Version Control and Resource Management) make versioning a viable option, for the first time;
and the IDE and intellisense improvements will definitely improve your productivity and
efficiency. Many of the other features are significant and long awaited improvements to some of
QTP’s central mechanisms, and they are mostly well executed.

However, weighing QTP 10 potential (at least for the IDE improvements) against the actual
features and changes, I am left with a strong sensation of a missed opportunity. This is not to
say that QTP 10 is not a worthwhile version, but that in my view it could’ve been great.

Page 19
© All rights reserved – Solmar Knowledge Networks
Here’s a concise “bottom line” review:

Area Feature Conclusion

QC integration Version Control and comparisons
(based on live demo).
Portability Save test with associated
resources
IDE To-Do comments
To-Do tasks
Variable auto complete and inner
object intellisense
Customize menus
Code block highlighting
Report Export to Doc, PDF, HTML
System Monitor
Natively add images to report
events
Jump to script line from report
step
Actions Dynamically add actions without
pre-attaching them to the test
Add, edit and overwrite actions
through the automation API
Bitmap See the actual-expected
Checkpoints difference bitmap
Write custom bitmap DLLs
Misc. Overall look and feel
Amazing. Completely transforms the CM
for the automation team.
Excellent solution to a very bugging
problem. Well executed.
Excellent collaborative and tasks
management tool.
Nice idea, but without the ability to link a
task to a specific line, I’d rather work with
Comments.
Nice and will certainly come in handy. The
bad implementation makes the whole
thing useless when it’s mostly needed (not
parsing library files or VB Script classes,
straightforwardly parsing functions, no
support for collections, etc.)
Works fine, and does exactly what it’s
supposed to.
Nice, but pales in comparison to other,
older, and cheaper IDEs.
Excellent. Still lacking an automated API for
exporting / saving / sending the results.
Great idea, Wonderfully executed. Will
probably become a must for almost every
project. Missing the ability to monitor
more than one process.
A true necessity, great implementation
Sounds cool, but meaningless if you’re
using a report function
Yes, Yes! YES! Works just as I expected
(which is good).
Excellent. Can’t way to start experimenting
with it. Power users will love this.
Greatly improves the usefulness of the
checkpoint report.
Seems unnecessary – serves power users
which I doubt use checkpoints. However,
very well documented, and easy enough to
implement. Well executed.
Runs quite fast, despite the added
features. Mostly looks updated and
professional.

Page 20
© All rights reserved – Solmar Knowledge Networks