This post is really a response to the post written by Pradeep on Traditional Software Testing Process
I saw that post as one where a tester proclaiming himself as an alternate to all those companies that have been cheating (by taking traditional testing approach) their clients and slamming the script based testing
I believe any tester who has experienced exploratory testing, enjoyed the freedom and witnessed results will accept ET as one good handy test methodology. As many ET tester do, I would love to read and hear a tester who practices and advocates testing on the lines of pre written test cases ONLY.
When I say that, I donot blame the testers entirely for readily nominating themselves to be caged by the traditional approach because I work with such testers day in and day out.
That’s exactly why I particularly felt relieved on reading his post because he seemed to have moved on with scripted testers to scripted testing and the reasons why testers its been widely used at the risk of clients not being served – see through the obvious.
Something that I try in everything I do, see thru why its happening. “Testers follow scripted testing methodology and further they advocate the same”, why would a tester do that? In my little experience, ignorance is largely the reason besides the business model we are in.
To elaborate more about the irony many of us share; lets keep ET out of equation for sometime.
Irony:
Even though the testers feel bored to run the test cases again and again, even though they like spending whatever time possible to play with the application (exploratory testing) which offers lot of challenges, learning, opportunity to find defects they continue running those test cases release over release.
Obvious:
Atleast with the small number of testers whom I interact with on a daily basis go thruthe above irony and I can safely presume that this would be echoed by all the services based organizations of all size.
See thru the obvious:
This is something that I attempt and expect the likes of you as well.
If a tester (whoz running the scripts) gets the freedom to do heuristics testing, I bet he will gallop this. The reason why they continue with the traditional approach is I guess not because of the awareness about the benefits of exploratory testing
Even though they know ET can help them unearth lot of defects, they don’t do it because the organization they work mandate them to do so ??
Lets RCA this
Do you really think the clients are blind and not capable to realize that they pay $9000 (even more as well) to have not even a single defect?
I saw that post as one where a tester proclaiming himself as an alternate to all those companies that have been cheating (by taking traditional testing approach) their clients and slamming the script based testing
I believe any tester who has experienced exploratory testing, enjoyed the freedom and witnessed results will accept ET as one good handy test methodology. As many ET tester do, I would love to read and hear a tester who practices and advocates testing on the lines of pre written test cases ONLY.
When I say that, I donot blame the testers entirely for readily nominating themselves to be caged by the traditional approach because I work with such testers day in and day out.
That’s exactly why I particularly felt relieved on reading his post because he seemed to have moved on with scripted testers to scripted testing and the reasons why testers its been widely used at the risk of clients not being served – see through the obvious.
Something that I try in everything I do, see thru why its happening. “Testers follow scripted testing methodology and further they advocate the same”, why would a tester do that? In my little experience, ignorance is largely the reason besides the business model we are in.
To elaborate more about the irony many of us share; lets keep ET out of equation for sometime.
Irony:
Even though the testers feel bored to run the test cases again and again, even though they like spending whatever time possible to play with the application (exploratory testing) which offers lot of challenges, learning, opportunity to find defects they continue running those test cases release over release.
Obvious:
Atleast with the small number of testers whom I interact with on a daily basis go thruthe above irony and I can safely presume that this would be echoed by all the services based organizations of all size.
See thru the obvious:
This is something that I attempt and expect the likes of you as well.
If a tester (whoz running the scripts) gets the freedom to do heuristics testing, I bet he will gallop this. The reason why they continue with the traditional approach is I guess not because of the awareness about the benefits of exploratory testing
Even though they know ET can help them unearth lot of defects, they don’t do it because the organization they work mandate them to do so ??
Lets RCA this
Do you really think the clients are blind and not capable to realize that they pay $9000 (even more as well) to have not even a single defect?
As I understand, the very human behavior pattern is to avoid risk.
By outsourcing, that’s exactly what we do. As an organization, I want to pass on the risk to someone else that can do my work with labels like “Cost”, “Operational reasons” and many more hopping to get a product of better quality then I can.
This challenge is in my opinion, comparatively less on the rest of the activities (except testing) in a software building process. We have a lot of tools, methodologies with very high percentage of repeatability is available. More than any other tool, “google” does magic there.
All these can not work with testing, because no matter how hard we try to with the likes of process, tools, re-usage (most misunderstood and misrepresented), testing is all about human intelligence (common guys, this is a gift after all).
When an activity like “testing” is turned over to a second party, especially third party companies, because of the very service outsourced model, it becomes imperative for the outsorucee to establish confidence in the client that they have the capabilities that are called to do the job.
Now it’s a two way situation; go with the abilities or competency of people (tester) or the organization (process)
It’s the practical difficult that a company cannot have expert testers ONLY in their payroll working across all the projects that are on flight. Even if an organization claim to have expert testers in their payroll, all the testers will not be internationally acclaimed or atleast have footprints outside the organization.
Alaaaas, now how do I convince my client that I can do this work?
Now the other way around is on the Process and this where the argument begins !!
Here is where the organizations, testers take the easy way out or probably the only way they think is available ie going behind certification, going Six Sigma, ISO standards and so on.
Processes naturally introduces the documents and hence the testers end up spending 9000 hours
I do not believe the clients that we are talking about are so blind to be made
intentionally bind . I know projects where the vendors end up loosing the contract on such instances (intentional blinding) and worst, had to pay the contract money.
So clients are the businessmen who have the caliber to see if they are going to be made blind so lets not shame them by labeling Ignorant
Either they get eventually get what they want or this is exactly what they want (the documents and process). This is exactly the challenge that testers of my kind face.
In that sense, I would label the likes of Pradeep, lucky because you have the luxury of saying “this is my approach and the best” and your clients must have been either buying or they don’t care about your approach but the results.
This naturally brings me to the approach that I could devise in my own little world with all the constrains a typical tester face.
I give what my clients what they want. Yes, you read exactly right, but I do in my way.
I give them test cases but I test my application based on the scenarios that I make out of the understanding I have about the application.
The basis of my approach involves a lot of discussion with the all the people involved in the evolution of the application;
and questions, questions and more questions.
I take the traditional script based testing to service by client with what they want and I mix that with conglomerate of scenario based testing (in other words behavior based testing), ET
By outsourcing, that’s exactly what we do. As an organization, I want to pass on the risk to someone else that can do my work with labels like “Cost”, “Operational reasons” and many more hopping to get a product of better quality then I can.
This challenge is in my opinion, comparatively less on the rest of the activities (except testing) in a software building process. We have a lot of tools, methodologies with very high percentage of repeatability is available. More than any other tool, “google” does magic there.
All these can not work with testing, because no matter how hard we try to with the likes of process, tools, re-usage (most misunderstood and misrepresented), testing is all about human intelligence (common guys, this is a gift after all).
When an activity like “testing” is turned over to a second party, especially third party companies, because of the very service outsourced model, it becomes imperative for the outsorucee to establish confidence in the client that they have the capabilities that are called to do the job.
Now it’s a two way situation; go with the abilities or competency of people (tester) or the organization (process)
It’s the practical difficult that a company cannot have expert testers ONLY in their payroll working across all the projects that are on flight. Even if an organization claim to have expert testers in their payroll, all the testers will not be internationally acclaimed or atleast have footprints outside the organization.
Alaaaas, now how do I convince my client that I can do this work?
Now the other way around is on the Process and this where the argument begins !!
Here is where the organizations, testers take the easy way out or probably the only way they think is available ie going behind certification, going Six Sigma, ISO standards and so on.
Processes naturally introduces the documents and hence the testers end up spending 9000 hours
I do not believe the clients that we are talking about are so blind to be made
intentionally bind . I know projects where the vendors end up loosing the contract on such instances (intentional blinding) and worst, had to pay the contract money.
So clients are the businessmen who have the caliber to see if they are going to be made blind so lets not shame them by labeling Ignorant
Either they get eventually get what they want or this is exactly what they want (the documents and process). This is exactly the challenge that testers of my kind face.
In that sense, I would label the likes of Pradeep, lucky because you have the luxury of saying “this is my approach and the best” and your clients must have been either buying or they don’t care about your approach but the results.
This naturally brings me to the approach that I could devise in my own little world with all the constrains a typical tester face.
I give what my clients what they want. Yes, you read exactly right, but I do in my way.
I give them test cases but I test my application based on the scenarios that I make out of the understanding I have about the application.
The basis of my approach involves a lot of discussion with the all the people involved in the evolution of the application;
and questions, questions and more questions.
I take the traditional script based testing to service by client with what they want and I mix that with conglomerate of scenario based testing (in other words behavior based testing), ET
4 comments:
hey Parthi,
One thought about the comparison of Script based and Exploratory testing. From organization's perspective, there is always a risk associated with loosing the tester who is supporting the application under test. And when application has all the scripts/scenarios/test cases in place, this risk is somewhat reduced. Because in this situation anyone (a new joinee for that application)can always has something to look for and get the required functional understanding.
But when this situation is mapped to Exploratory Testing, there are no such things so the risk associated increased by many folds.
Testing as an "excercise" should not be individual dependent. In case of ET, it comes down to individuals because this is more intution based.
May be a healthy mix of both conventional and ET can give us the best results in terms of "Test Effectiveness".
I am not sure if anyone has ever tried to implement CMMI or any other worldwide recognized standard process using ET and as far as I understand these processes, in case of ET it won't be possible at all. any thought?
Rahul
One thought about the comparison of Script based and Exploratory testing. From organization's perspective, there is always a risk associated with loosing the tester who is supporting the application under test. And when application has all the scripts/scenarios/test cases in place, this risk is somewhat reduced. Because in this situation anyone (a new joinee for that application) can always has something to look for and get the required functional understanding.
These are simply barriers or blocks created by “God Knows who”, may be by the people who underestimate the human intelligence or afraid of the potential a human brain.
Why do you think a tester need test cases of another tester to test a software as long as the tester understands the software and the mission or goal.
How do we make sure that we are not passing on the mistakes of one tester to the other while doing the exercise? (Passing scripts from one to the other)
People and their ability to think is the asset to any organization and its certainly more in Software Testing, because Software Testing is all about one’s ability to see thru and question the reason.
By any of the measure proposed by the traditional scripted methodology, this can not be taken away from a tester. If we still convince either ourselves or the clients that we can manage this via having scripts in place, then as Pradeep says we are cheating the clients (either intentionally or unintentionally).
But when this situation is mapped to Exploratory Testing, there are no such things so the risk associated increased by many folds.
Leave alone ET, as testers if we can not produce a report of what we tested then I would neither call the activity as a testing nor the person as tester. It can be anything else.
And on ET, exploratory testing is as Cem Karner defined “Simultaneous test design and test execution”.
At the end of the testing session, the tester should be able to produce the report on testing that he performed and with the tester should also be able to summarize his opinion on the quality of the application, having the test report as proof.
I am not sure if anyone has ever tried to implement CMMI or any other worldwide recognized standard process using ET and as far as I understand these processes, in case of ET it won't be possible at all. any thought?
Why should the work that we should be certified by any third party organization that has no idea about the work, practically day to day challenges that as tester we face. All they do is assess our work based on some documents that are created ?!!!!
Why do you think a tester need test cases of another tester to test a software as long as the tester understands the software and the mission or goal.
How would someone pass his/her knowledge to new joinee who has no idea of the AUT?
And as you mentioned about Cem Karner, I referred to some of his presentations available at http://www.kaner.com/articles.html.
At slide 136 ["A tutorial in exploratory testing." [SLIDES] QAI QUEST Conference, Chicago, April 2008"], he mentioned ET -Interpretaion
"Interpretation: What do we learn from program as it performs under our test
–about the product and
–about how we are testing the product?"
It means in ET, one learns while he/she starts executing
On slide number 145 he said "ET is not quicktesting" though he agreed to list it under "Areas of Controversy".
Also on slide 159 he said
"ET is a way of Testing
-We take advantage of what we learn to design better tests and
interpret results more sagely"
Means we still design tests or rather re-usable tests in ET.
He also mentioned "How to assess individual tester performance" under "Areas of Ongoing Concern". This is one of the concerns mentioned by him.
I am not questing capabilities of ET but its an attempt to understand ET thoroughly and improve myself as a tester.
And regarding CMMI and certifications, one has to have that much credibility first to certify his/her own work and if thats not the case, then some organizations can do that after assessing the required information. Because as per Cem Karner, assessing individual performance is still a concern in ET. :)
Happy testing.
How would someone pass his/her knowledge to new joinee who has no idea of the AUT?
Let me re-phrase the question which can perhaps help us see thru the obvious.
There is no other way to pass on tester knowledge to other besides passing on test cases?
How about the knowledge on the approach taken by one to test the application?
How about the all the different contexts the application is to be used and the users?
At slide 136 ["A tutorial in exploratory testing." [SLIDES] QAI QUEST Conference, Chicago, April 2008"], he mentioned ET -Interpretaion
"Interpretation: What do we learn from program as it performs under our test
–about the product and
–about how we are testing the product?"
It means in ET, one learns while he/she starts executing
In my opinion, this is the strength of ET. As a methodology, ET provides large room for the tester to do improvisation on the tests done.
And regarding CMMI and certifications, one has to have that much credibility first to certify his/her own work and if thats not the case, then some organizations can do that after assessing the required information. Because as per Cem Karner, assessing individual performance is still a concern in ET. :)
Happy testing.
The test reports that are published should be scrutinized by the stakeholders and that in itself would bring the Credibility. While saying that the scrutiny should refrain from creating path in other word barriers for a tester/organization in doing their job.
Post a Comment