Test Lab hard at work
The test tool panoply

There are now a multitude of tools that test teams can use throughout the test process and across all types of testing. The question is no longer “is there a tool that can do this?”, but rather “which one do I pick?” Testers now must select the right tools and ensure that they will provide the most benefit. We must consider whether we require integrated enterprise solutions, such as those offered by HP or IBM, or individual point solutions that may have to integrate with other tools already in use. What we end up selecting is usually driven by how quickly something needs to be set up or more often by the budget available. As test teams often do not have large budgets set aside for tools, we see lots of freeware and open source tools being implemented.

Test management tools

For test management most people still think of HP ALM (which includes the widely used Quality Center) or IBM’s Rational ClearQuest. However we are increasingly seeing test teams using HP ALM alongside development teams who are managing their delivery using Jira. This can present the project team with challenges around both process and tool integration. There are integration solutions such as Tasktop Sync and Orasi Software’s Jira Bridge. Other tools have appeared such as Zephyr Enterprise and the Zephyr for Jira add in editions that sync directly with Jira.

The longer established test management tools are often criticised for not supporting Agile development methodologies. If you already have HP ALM but need to respond to Agile then look at HP’s own Sprinter for Agile. However, if you are limited to something free and open source for managing your software testing in Agile then the Tarantula might be worth checking out.

Defect management tools

Incident management capabilities are again provided by established bigger players’ tools. A low cost alternative is provided by Bugzilla, which is still very popular with many test teams. A free hosted service is provided by Damn Bugs which also includes graphical reporting and screen capture capabilities. If you’re looking for a built for Agile alternative then take a look at the Yodiz an online project management tool that will even allow you to track bugs using it’s mobile app.

Functional testing tools

For test execution automation a lot of non-programmer testers still think of HP’s UFT, formerly QTP, when performing GUI or web application testing. Ranorex offers very similar capabilities for desktop, web and mobile application testing. If you are looking for something free and open source to automate testing of web browsers/applications then consider Selenium. SoapUI is another free and open source cross-platform functional testing solution that supports all the standard protocols and technologies.FitNesse is another popular tool for automation.

Data management tools

Data comparison testing can be undertaken by more technical testers making use of functions provided in data management tools such as Pentaho Data Integration (Kettle), IBM’s Infosphere Optim and Curium. There are also open source tools like Bucketsoft’s Data Comparisons that allow testers compare data between two SQL database tables quickly and easily.

Load and performance testing tools

For performance testing, the most well known tools are HP LoadRunner, now rebranded Performance Center, and Rational Performance Tester. However, Microsoft VSTS, though it does not cover as many protocols, is a lower cost alternative. SoapUI is a free and open source and lets you create even the most advanced Load Tests quickly and easily.

Reporting issues, status and progress

Whether using an integrated tool set or point solutions a key issue is how we present test progress and defect information to the stakeholders. They often need to make the important decision of what goes live when and so require clear and reliable information from the test process. Not all tools allow testers to provide this easily or in a way that is immediately understood by key stakeholders. Test teams often have to devise various methods to interpret technical information, produce data extracts or enhance metrics and measure that come out of the box.

Tool selection

We’ve only covered a few of the tools available in each of these categories and there are lots of excellent testing tools available not mentioned here. There are also lots of other categories with specialist tools that provide real value in testing. For example, BDD tools (Behaviour Driven Development) such as Cucumber and Capybara; or continuous integration tools like Jenkins.

If you are choosing a tool to help in testing, do take the time to find out the options you have. And take seriously the option of not using a tool. Research it yourself or get help from an organisation that has a wide knowledge of testing tools and is independent of tool vendors. There are lots of criteria you can use for selection (for example, what it can do, how easy it is to use, how easy it is to set up and maintain, how well it integrates with other tools) and how well these fit with your team; the support available and the cost. Cost should be cost of ownership rather than just up front licensing costs. We will look in more detail at tool selection in a separate post, including handling ROI if you need to present a business case. Above all things, select and implement a test tool that will provide demonstrable benefit to you and measure and report that benefit.

Contact acutest