Tuesday, February 9, 2010

Automated Test developers are not “Checkers”

May be because of the uncalled for importance given to the “automation” part of the testing made the automated testing to be a target for lot of “testing” enthusiasts.

I’m very passionate about being a tester which includes automation testing and non functional testing. One fine day, I was going through some blogs and forums on testing and happened to read a forum which called the automation testers “checkers” (I know Michael Bolton called automation testing as “checking” but not automation testers)

That made me bat for the automation test developer avatar of mine via this post.

Checker: - It is the code or script that’s written does checking not the person who wrote the code. Now the checker is actually the code not the automation tester, is it not?

Automation tester:- A person (remember testing can be done only by the human being as it needs common sense to think and apply) who tests automation. Do we mean the code or the script as “automation” here? May be …

But the dictionary says Automation is an act or process of automating or the state of being automated

I wonder if anyone does it i.e. testing the process of automating because I get to hear “Automation Process”, (funny, A process of process of automating. My head is spinning now)

So I thought of calling the people involved in Test automation as “Automated Test Developers” and the tests that are run via coded scripts Automated Tests. Doesn’t that sound better, ohhh it’s no invention!!!

Automation and manual testing (I mean sapient testing, for clarity sake permit me to use manual) are like Man and Women. They can live separate but will face challenges such as time lines and coverage by manual testing and validity of testing, lack of human sense by automated testing. But when combined together with correct complexion in the right context, they complement each other and can yield better results.

Both call for equally good analytical skills, logical thinking. One should understand that testing is a support function for product development and automation is a support function for testing such as running similar tests for large data sets on a controlled environment in quick time.

For sake of automation, automating everything is just not good as its bound to defy the goal of testing.

We test a large order generation application (capital market) and I don’t see any point in automating all the test scenarios even though I run these tests release over release purely because the automated tests are not intelligent enough to test the application for all the reasons why the coded expected result is not met. In the end, on the name of validating those results if I endup testing the scenarios which are automated, what I have done is spending time in developing scripts to test it and then testing it all over again myself.

On the other hand, I had to do data validation on a migration project where we were moving from one technology to another on autosys jobs and database which have a standard input and output data set.

Here it’s the verification that we had to between the migrated-from data to migrated-to data. We used a set of tools to do this testing and hence we used to verify almost 20 different reports per day and we completed testing in a span of 2 weeks.


2 comments:

Rahul Gupta said...

Good post Parthi,

I remember reading the post by Michael on Test Automation. There is an Article "Test Automation Snake Oil" by one of the "Testing Experts" if I may call them so. I believe that the basic idea of Test Automation is to have a larger coverage in lesser time. The naysayers say "Why do you want to run same test cases again and again when you won't have any new defects getting out of it?"
I think that is what we call Regression testing or in simple terms "Testing around defect re-testing".
One last point, I agree that automation cannot replace human testers but it is certainly not a Snake Oil.

Thanks,
Rahul

Unknown said...

Hi Parthi,

Not only it helps doing repetative job like regression testing but also reduces the time required for each regression. The ultimate RoI on the use of Automation for large organization which do quarterly enhancements is tremendous.

Thanks ,
Taral Shah