Before we understand why we do context driven testing, we must understand what is the traditional way of testing and what it really means by context driven testing.
The Traditional Way of Testing
For a long time period, debates have been going around the software testing community over the right way to do testing. Various writers, speakers, groups, and institutions touted their approaches as being the best practices. Cem Kaner, first to notice that “best practices” coming from academia didn’t work so well when you try to apply them in industry. Those best practices published in books by people who worked in the bank business didn’t work so well for the people in Silicon Valley. And those best practices for mass-market commercial software went against the medical business. Based on those conclusions, they were declaring that testing should be thus and so. It is not that their finding were wrong, they might have been right for their own contexts. The problem is that it is not universally applicable to all domains.
Birth of Context Driven Testing
As the time went by more people were facing the same problem. The reasonable people suggested that there is no best practice and it can not be applied to all domains. Standards for a medical solution will not work for a computer or mobile game development industry. People like Cem Kaner, James Bach and Bret Pettichord who recognized these issues went on to declare a set of principles based on the idea that if you want to add value to your solution by your testing, then you have to consider the context first, before anything else.
The Context Driven Testing
The Seven Basic Principles of the Context-Driven testings are:
- The value of any practice depends on its context.
- There are good practices in context, but there are no best practices.
- People, working together, are the most important part of any project’s context.
- Projects unfold over time in ways that are often not predictable.
- The product is a solution. If the problem isn’t solved, the product doesn’t work.
- Good software testing is a challenging intellectual process.
- Only through judgment and skill, exercised cooperatively throughout the entire project, we are able to do the right things at the right time to effectively test our products.
Principles in Action
- Testing groups will serve the project for testing related services, they do not run the project.
- Testing strategies can be differ according to stakeholders objectives. Objectives can be developing, qualifying, debugging, investigating or other kind of services related to it.
- Different test groups may have different missions thus establishing a standard practice for a mission will not work
- Invalid metrics are dangerous
- The value of a test case is in its ability to be informative for reducing uncertainty.
- A passed test does not mean that it may not fail in ways that you and your script is ignoring.
- Automated testing is not manual testing done automatically.
- Different type of bugs will reveal in different scenarios. Testing technique should focus on different aspects of different scenarios.
- Testing is worthwhile to satisfy their stakeholders requirements.
So, Why Context Driven Testing?
Before asking this you must make sure your answers to the following questions are all affirmative:
- Do you value more in individuals rather than their interactions over processes or tools?
- Do you value more in seeing working software over documentation?
- Do you value more in collaborating with your customers (including development) over negotiating contracts (and specs)?
- Do you value more in responding to change over following the plan?
Some people believe that you should refuse to test until you got a clear, complete, up-to-date, and unambiguous specification. Is it really applicable in industry? Think about latest fashion of industry prototyping, extreme programming, agile etc.
Now, as it’s been said, people may ask “So, you don’t believe in documentation?” Not true; we do believe in documentation. But we believe in communication more. We believe conversation more than documentation versions because it is more clear, faster and effective.
Some people say “Exploratory testing? So that’s manual testing, you don’t believe in test automation?” Not true; we do believe in automation as a tool. We favour interacting with the machine as the users of the product do, but we also love using automation for things that automation can help us with.
Context driven testing is an approach not a technique. Do not use it as oracles or standards being used in traditional testing. Change oracles according to your context and keep asking yourself WHY?Contributor: MD. Wasiqul Huq, Software Engineer (QA), Nascenia IT