ICICI Online Banking e-Statements

Automated Test developers are not “Checkers”

May be because of the uncalled for importance given to the “automation” part of the testing made the automated testing to be a target for lot of “testing” enthusiasts.

I’m very passionate about being a tester which includes automation testing and non functional testing. One fine day, I was going through some blogs and forums on testing and happened to read a forum which called the automation testers “checkers” (I know Michael Bolton called automation testing as “checking” but not automation testers)

That made me bat for the automation test developer avatar of mine via this post.

Checker: - It is the code or script that’s written does checking not the person who wrote the code. Now the checker is actually the code not the automation tester, is it not?

Automation tester:- A person (remember testing can be done only by the human being as it needs common sense to think and apply) who tests automation. Do we mean the code or the script as “automation” here? May be …

But the dictionary says Automation is an act or process of automating or the state of being automated

I wonder if anyone does it i.e. testing the process of automating because I get to hear “Automation Process”, (funny, A process of process of automating. My head is spinning now)

So I thought of calling the people involved in Test automation as “Automated Test Developers” and the tests that are run via coded scripts Automated Tests. Doesn’t that sound better, ohhh it’s no invention!!!

Automation and manual testing (I mean sapient testing, for clarity sake permit me to use manual) are like Man and Women. They can live separate but will face challenges such as time lines and coverage by manual testing and validity of testing, lack of human sense by automated testing. But when combined together with correct complexion in the right context, they complement each other and can yield better results.

Both call for equally good analytical skills, logical thinking. One should understand that testing is a support function for product development and automation is a support function for testing such as running similar tests for large data sets on a controlled environment in quick time.

For sake of automation, automating everything is just not good as its bound to defy the goal of testing.

We test a large order generation application (capital market) and I don’t see any point in automating all the test scenarios even though I run these tests release over release purely because the automated tests are not intelligent enough to test the application for all the reasons why the coded expected result is not met. In the end, on the name of validating those results if I endup testing the scenarios which are automated, what I have done is spending time in developing scripts to test it and then testing it all over again myself.

On the other hand, I had to do data validation on a migration project where we were moving from one technology to another on autosys jobs and database which have a standard input and output data set.

Here it’s the verification that we had to between the migrated-from data to migrated-to data. We used a set of tools to do this testing and hence we used to verify almost 20 different reports per day and we completed testing in a span of 2 weeks.

(Magic) Reading List in my LinkedIn profile

I have 16 books in the reading list in my linkedin profile. Today I noticed something strange.

1. The link see all books on Parthiban's list(1)


2. I re-loaded the page by clicking on the Profile link and I see the same page loaded with See all books on Parthiban's list (16)


3. I continued by cliking on the link, and the page listed first 10 of 16 books with two page pagenation,


4. Clicked on link and the page listed the second set ie last 6 books


5. Clicked on button and what I see is a page with 1 book, Fountain Head. The reset disappared.

I couldnt reproduce this consistently, but with little patience I could see this behaviour happening again. Still to investigate on the possible reason.

If you have seen this behavious before, do share your exp

Processes Mythology

CMMi has been the buzz word at all levels at my firm for about 10 months now. There have been a lot of discussions, presentations and training sessions on the importance of having “Standardization” and “Predictability” of the activity that we do, be it development, testing, production support , so on and so forth.

This post is my experience with some the models around Data Collation, Analysis & Interpretation of Standard and Predictable testing process.

Processes remove Person-Dependency and improve Efficiency:

“This was one of the key benefits of having a defined processes in place” as was explained to me during one of the training sessions on processes.
I asked the trainer, “how do you say this?” and
the answer was, “you work on a project and if you do not follow the process (i.e. documentation) properly, when you leave the project team or the firm, then the knowledge goes with you. By bringing in the process, the necessary of the person’s presence is removed and the new person coming in will have a shorter learning curve as the documents will be readily available to her”.

I asked her, “What do you mean by “properly”?” and to that she said “You should prepare the document in such a way that it covers the matter in full”
I think a huge bottom line is being missed. “Hellloooo, Having all said and done, it’s the very people who need to do anything !!!!! “.
My take on this

· Never can a person express 100% of what one has in mind. Let’s assume conservatively that we loose 15% in translation (remember we are ignoring the environmental facts when the person writes), this leaves us with 85%

· Never a person can understand 100% of what is written in a document. Let us assume (conservative again) that we loose another 15% while reading, this leaves us with 72.25%

· Never a person can synthesize 100% of what is understood. Lets (conservative again) assume that we loose 20%, this leaves us with 57.8%

We have less than 60% of the understanding of the project one had when the person moves out of the project. Remember this is just what the out-going person “understands” and NOT his knowledge (am still understand what and how knowledge management happens).
Remember; we haven’t considered few highly influential factors in the above conjuncture. They can have both positive and negative impact; the list can go quiet long from knowledge to environment.

Below are the few that I could think of
· Writing skills of the out-going person in a foreign language (I know no company allowing people to write in their mother tongue in India)

· Ability of the incoming person in understanding a subject written in foreign language

· Prior knowledge/experience/knowledge of the incoming person on the similar subject/project

When I say this, I agree and accept that documentation is required to let myself know what I have done yesterday and the day before and if this documentation can help someone else to understand the project better, it’s a great. However, why so paranoid about “documentation” of EVERYTHING done by a person!!

With this premise, am still to understand how processes remove people dependency and improve efficiency. Can you help me please?

“Certified” thinking

I am a certified tester. I hold CSTE certification.

When I read the text (of course of myself) “Certified Software Testing Engineer”, I always wonder what I mean by that.

Doesn’t it mean that I don’t have my own mind to think and do testing but I learnt it (not practiced as none of the certifications are based on practical experiments), rather copying (legitimately) or re-using (famously) some one else’s idea?

During the early part (comparatively) of my career, I chose to sit for CSTE because

I wanted to prove a point to my managers that I am the best in the lot
I wanted to differentiate myself
I wanted to see myself in the newsletter (internal) and magazines (that used to carry the successful CSTE names with photographs at that time)
Company reimbursed the money that I spent for registration so I was anyway not spending money from my pocket
More than all the above, I wanted a motivation and a reason to read something about Testing

I registered myself, picked up the CBOK and started reading. I should admit there are some favorite chapters of mine in the book which I enjoyed reading. Eventually, I did clear the paper.

I can confidently say that the reading I did, the discussions I had with fellow CSTE aspirants certainly helped me in understanding a lot of details. If someone cites this as a reason to sit for the exam I wouldn’t give too much a piece of my mind.

A step further, in a way I would approve CSTE to some extent (not 100% though) because as testers we do have a 50% chance to write what we think is correct with the context we understand in the descriptive papers which I can safely call a kind of experimental based (atleast that’s what I did).

But the reality is that there are too many who use those CBOK as Bible or Bhagwat Geeta for testing principles. Much worse,
I saw a lot of people using certification as a token to call themselves as tester without bothering about either practicing or even an attempt to do practice
I saw people with no experience getting job by faking about it in big companies because they held a certification


Certifications compel the testers to a very dangerous extent, that we always tend to look for one CORRECT ready-made answer for all questions. Certifications restricts us from the thought process of,

All the options given could very well FIT
NONE of the options given (to choose from) may NOT fit
Dangerous of all is, it tunes the certified minds to an extend to look for even “Above All” or “None of the “Above” as answers when there isn’t clarity on what to choose from !!

We get accustomed to always look-for/think of a “Point-blank” answer for all questions, for contexts and certifications go out of their way to advocate and spread this at full throttle. A very simple example is one of the polls that I happened to see in an internal website, eventually ended up as post.

Reason to test software before releasing it is:
To find as many bugs as possible before release
To make a decision when the software is of good quality to release
To confirm that software will work after the release
To give information for a risk based decision about release

A classic question that appears in certification exams. I, to some extent understand that we still as a community yet to get matured to understand and think in contexts’. The reason why I felt all the more sorry for the certified testers is that the certifications have influenced so much, so that the author failed to see that “ALL” can also be an option if not an answer.

One of the eye openers for me was “A difficult testing quiz” in Pradeep’s Blog (that’s how I started reading his posts). I honestly attempted all the questions and the result was a lesson. That’s when I felt confident, that I don’t have to always worry about being correct.

There are a good number of testers who are trying to win credibility for “testing” as an activity and for testers. Meanwhile it’s disheartening to see there are large number of people who are still comparing/writing/discussing/arguing (??!!), if CSTE is good or ISTQB.


PS: CSTE’s CBOK is really a subset (edited) of William Perry’s book on Software Testing, which I read fully later and I think a good one to read.

Traditional Software Testing Approach and 9000 hrs

This post is really a response to the post written by Pradeep on Traditional Software Testing Process
I saw that post as one where a tester proclaiming himself as an alternate to all those companies that have been cheating (by taking traditional testing approach) their clients and slamming the script based testing

I believe any tester who has experienced exploratory testing, enjoyed the freedom and witnessed results will accept ET as one good handy test methodology. As many ET tester do, I would love to read and hear a tester who practices and advocates testing on the lines of pre written test cases ONLY.

When I say that, I donot blame the testers entirely for readily nominating themselves to be caged by the traditional approach because I work with such testers day in and day out.

That’s exactly why I particularly felt relieved on reading his post because he seemed to have moved on with scripted testers to scripted testing and the reasons why testers its been widely used at the risk of clients not being served – see through the obvious.

Something that I try in everything I do, see thru why its happening. “Testers follow scripted testing methodology and further they advocate the same”, why would a tester do that? In my little experience, ignorance is largely the reason besides the business model we are in.


To elaborate more about the irony many of us share; lets keep ET out of equation for sometime.

Irony:
Even though the testers feel bored to run the test cases again and again, even though they like spending whatever time possible to play with the application (exploratory testing) which offers lot of challenges, learning, opportunity to find defects they continue running those test cases release over release.

Obvious:
Atleast with the small number of testers whom I interact with on a daily basis go thruthe above irony and I can safely presume that this would be echoed by all the services based organizations of all size.



See thru the obvious:
This is something that I attempt and expect the likes of you as well.

If a tester (whoz running the scripts) gets the freedom to do heuristics testing, I bet he will gallop this. The reason why they continue with the traditional approach is I guess not because of the awareness about the benefits of exploratory testing
Even though they know ET can help them unearth lot of defects, they don’t do it because the organization they work mandate them to do so ??

Lets RCA this
Do you really think the clients are blind and not capable to realize that they pay $9000 (even more as well) to have not even a single defect?

As I understand, the very human behavior pattern is to avoid risk.

By outsourcing, that’s exactly what we do. As an organization, I want to pass on the risk to someone else that can do my work with labels like “Cost”, “Operational reasons” and many more hopping to get a product of better quality then I can.

This challenge is in my opinion, comparatively less on the rest of the activities (except testing) in a software building process. We have a lot of tools, methodologies with very high percentage of repeatability is available. More than any other tool, “google” does magic there.

All these can not work with testing, because no matter how hard we try to with the likes of process, tools, re-usage (most misunderstood and misrepresented), testing is all about human intelligence (common guys, this is a gift after all).

When an activity like “testing” is turned over to a second party, especially third party companies, because of the very service outsourced model, it becomes imperative for the outsorucee to establish confidence in the client that they have the capabilities that are called to do the job.

Now it’s a two way situation; go with the abilities or competency of people (tester) or the organization (process)

It’s the practical difficult that a company cannot have expert testers ONLY in their payroll working across all the projects that are on flight. Even if an organization claim to have expert testers in their payroll, all the testers will not be internationally acclaimed or atleast have footprints outside the organization.

Alaaaas, now how do I convince my client that I can do this work?

Now the other way around is on the Process and this where the argument begins !!

Here is where the organizations, testers take the easy way out or probably the only way they think is available ie going behind certification, going Six Sigma, ISO standards and so on.

Processes naturally introduces the documents and hence the testers end up spending 9000 hours

I do not believe the clients that we are talking about are so blind to be made
intentionally bind . I know projects where the vendors end up loosing the contract on such instances (intentional blinding) and worst, had to pay the contract money.

So clients are the businessmen who have the caliber to see if they are going to be made blind so lets not shame them by labeling Ignorant

Either they get eventually get what they want or this is exactly what they want (the documents and process). This is exactly the challenge that testers of my kind face.

In that sense, I would label the likes of Pradeep, lucky because you have the luxury of saying “this is my approach and the best” and your clients must have been either buying or they don’t care about your approach but the results.

This naturally brings me to the approach that I could devise in my own little world with all the constrains a typical tester face.

I give what my clients what they want. Yes, you read exactly right, but I do in my way.

I give them test cases but I test my application based on the scenarios that I make out of the understanding I have about the application.

The basis of my approach involves a lot of discussion with the all the people involved in the evolution of the application;

and questions, questions and more questions.

I take the traditional script based testing to service by client with what they want and I mix that with conglomerate of scenario based testing (in other words behavior based testing), ET

Secrets of a Buccaneer-Scholar: Must read not just for Testers

I’ve been trying to write about this latest book that I read for almost 4 days now and finally I got time @ 1.50 am today.

Acknowledgement
Thanks to Pradeep, for writing about this book. I know nothing of James would go unnoticed by him. So by visiting Pradeeps’ posts, I get to read many good ones besides his and this was the best of all so far.

About the Book:
I started reading this book after a little technical tussle in downloading the free copy on a Saturday evening and my excitement had already started. I have to admit that the book was so compelling that I had to make some time during by office hours (thought I was running pretty busy) even if that means I had to stay little late in office.

How did I relate to the Buccaneer-Scholar
I was just going thro the first few pages of the book and a specific lecture I attended during my masters degree flashed right in front of my eyes.

My 4th semester period, a week day, around 5.30 pm.
The then Head of the Department of Computer Applications was in the class, not explaining anything around software but trying to understand why there was a good number of students (~30%) bunking the class consistently (he was actually talking about myself and some of my friends). As he was addressing, I couldn’t wait anymore and told him “the time that we spend in the class room (we were spending almost 7 hours a day attending different lectures) were of no use (no use == we werent hearing/going thru/reading about anything other than the contents in the text books prescribed) and I could learn more by just spending couple of hours in the library and with technical magazines”

After that day for all obvious reasons, I had to face consequences. Because I had questioned the very reason of the teaching that he (HoD) thought was happening and of course the rest of the faculty members thought process about teaching.

This instance actually put me and some of my friends in a spot. We had to prove that we were indeed better of without those lectures. To prove a point we had to work harder hence followed more hours in library, more hours reading about lot of stuff which ended up as a paper on VoIP by myself and one of my friends during our 5th semester. This lead us to become winners in one of the technical symposiums (on September 24th 2002) in paper presentation category. That was first of its kind in my college.

Back to present; I wish I had such a book to read when I was a student. I wouldn’t say I would have quit the college . But it would have certainly made my outlook about learning far better.

I recommended this book to some of the testers that I work with; but that’s before I read the book full. Now that I have read the book completely, I wish every student gets a chance to read this book and make the correct sense of it.

Thanks very much James - for penning down your experiences and few others and

(again) Thanks Pradeep – for your post on the book (that’s how I came to know about it)