Thursday, February 4, 2010

Test Automation

What is Test Automation ?

Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process

Overview

Although manual tests may find many defects in a software application, it is a laborious and time consuming process. In addition it may not be effective in finding certain classes of defects. Test automation is a process of writing a computer program to do testing that would otherwise need to be done manually. Once tests have been automated, they can be run quickly. This is often the most cost effective method for software products that have a long maintenance life, because even minor patches over the lifetime of the application can cause features to break which were working at an earlier point in time.

There are two general approaches to test automation:

1) Code-driven testing:- The public (usually) interfaces to classes, modules, or libraries are tested with a variety of input arguments to validate that the results that are returned are correct.

2) Graphical user interface testing:- A testing framework generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct.

Test automation tools can be expensive, and it is usually employed in combination with manual testing. It can be made cost-effective in the longer term, especially when used repeatedly in regression testing.

One way to generate test cases automatically is model-based testing through use of a model of the system for test case generation, but research continues into a variety of alternative methodologies for doing so.[citation needed]

What to automate, when to automate, or even whether one really needs automation are crucial decisions which the testing (or development) team must make. Selecting the correct features of the product for automation largely determines the success of the automation. Automating unstable features or features that are undergoing changes should be avoided

What to test

Testing tools can help automate tasks such as product installation, test data creation, GUI interaction, problem detection (consider parsing or polling agents equipped with oracles), defect logging, etc., without necessarily automating tests in an end-to-end fashion.

One must keep following popular requirements when thinking of test automation:

a)Platform and OS independence
b)Data driven capability (Input Data, Output Data, Meta Data)
c)Customizable Reporting (DB Access, crystal reports)
d)Easy debugging and logging
e)Version control friendly – minimum or zero binary files
f)Extensible & Customizable (Open APIs to be able to integrate with other tools)
g)Common Driver (For example, in the Java development ecosystem, that means Ant or Maven and the popular IDEs). This enables tests to integrate with the developers' workflows.
h)Supports unattended test runs for integration with build processes and batch runs. i)Continuous Integration servers require this.
j)Email Notifications (Automated notification on failure or threshold levels). These may be the test runner or tooling that executes it.
k)Support distributed execution environment (distributed test bed)
l) Distributed application support (distributed SUT)

Software Test Automation is the process of automating the steps of manual test cases using an automation tool Or utility to shorten the testing life cycle with respect to time…

When application undergoes regression, some of the steps might be missed out or skipped which can be avoided in Automation…
Automation helps to avoid human errors and also expedite the testing process…
To implement the Test Automation detailed planning and effort is required
Automation saves time and effort which results in reduction of the Test life cycle…

Benefits of Automation

- Consistency of Test Execution
- Reducing cycle time of regression test cycles
- Data driven testing
- Repeatability
- Coverage
- Reliability
- Reusability of test wares

Automation life cycle is a subset of the entire test life cycle…
Automation planning can be initiated in parallel to the test planning phase…
Factors to be considered in automation planning,
Stability of AUT (Application under test)
No of regression cycles to be performed
Compatibility of App platform with testing tools
Cost benefit analysis (ROI)
Availability of skilled resources

When Automation is applicable?

1) Regression Testing Cycles are long and iterative.
2) If the application is planned to have multiple releases / builds
3) If it’s a long running application where in small enhancements / Bug Fixes keeps happening
4) Test Repeatability is required

No comments: