Automation testing tools…. comparative study

Introduction:

Automating testing GUI based applications entails carrying out few or all of the following tests (Web tests, Database tests, Image tests, Integration tests and Object tests) depending on their relevance.

This necessarily requires tools with abilities to record and playback, allow for data functions, recovering from errors, have facility to map objects, have an extensible language (with environment support) and above all – be easy to use.

Features Comparison:

It isn’t easy to recommend any ONE tool as the selection will have to be backed by a thorough understanding of the task at hand as well as other constraints such as cost, schedule etc. Apart from the technical abilities that need to be evaluated, the purchase decision must also be evaluated in terms of the long-term purpose it could serve. Following factors could be considered before reaching such a decision.

1) Price – If price is a constraint, then we have found that Visual Test, Robot and QARun tend to be at the more economic end of the scale. Apart from Visual Test, the other two are comparable to Silk and Win Runner in most areas.

2) Integrated with development environment – If price is not an issue and you want a comprehensive package that may require linking to development tools then Compuware and Rational may be preferred.

3) Web testing – for web testing we would tend to evaluate Win Runner (or QTP), Silk Test and Robot first in that order.

4) Regression testing – If the purpose of automation is also to assist with regression testing at a later stage, you will be creating huge regression packs for several projects possibly on various platforms. We would look at Silk Test, Win Runner and QARun in that order as these tools have superior facilities for maintaining scripts in the GUI MAP or similar facilities as well as baseline recovery systems. As

5) Customer Support – If you are going to require a lot of professional services support or helpdesk support then Mercury and Compuware are preferable choices.


Tool’s strengths and weaknesses:

Rational:
Strength: - Economical, Lots of supporting tools, good extensible language through VB, good data creation facilities, good online community.
Weakness: - Customer Support, language is a little basic and a bit confusing with tools breakdown e.g. Visual Test, Team Test, Test studio, Test Factory, etc.

Mercury:
Strength: - Possibly the most popular test tool, skill set available in plenty, good support, good online community, Good cross browser support.
Weakness: High cost.

Compuware:
Strength: - Strong professional services, good integration with development products, Economical.
Weakness: - Poor online community, Product breakdown

Segue:
Strength: - Good development language, good online community, recovery system.
Weakness: - Helpdesk, High cost.


Conclusion:

So which tool is best?

Well the winner is
1. Visual test
2. Robot
3. Win runner (or QTP)
NOTE: Quick test pro is easier to use and implement for both technical and non-technical testers in comparison to Win runner. QTP offers many features that are found in win runner, but are easier to use and QTP is economical.
4. Silk test
5. QA Run.

*** Test complete automation tool: is another option if cost is a big issue

An ideal tool should have the following characteristics:

1. As cheap as Visual Test,
2. Good data creation facilities as Robot,
3. Good cross browser and wizardry as Win Runner or QTP.
4. Flexible and powerful language as Silk Test and
5. A central repository and text facilities as QA Run.

------------------------------------ 0 ----------------------------------------