 |
||
  |
||
| Testing - How Much Is Enough? | ||
| There
are days when I think that I spend more time testing my user’s system
and in proving that I have tested it than I did in actually building it.
I know that I not only need to minimize errors in the system, but to prove
to a variety of entities that I really did test it. These days, we seem to be in an endless discussion of the different levels of testing and the names we should call each of them. Additionally, we seem to be at a loss when we come to the issue of what to do to accomplish the different types of testing. Let us consider a few types of testing. This is a rough sketch, as I realize that there are more or less levels and a variety of names, depending on your own organization: Unit Testing: Testing the “units” of code. This is also what we usually call “white box” testing, where we must test using a deep knowledge of what the code does. Some organizations require that formal Unit Test Plans be written. In others, it is considered sufficient that the programmer states that they did do this testing. Unit testing is only as good as the person that is doing the testing. If the person who wrote the code is not very good and is also the person unit testing it, it is not much of a test. When the writing of the code is separated from doing the test, the outcome of the testing is better; however, doing a separate unit test is a luxury only the larger projects can usually manage. Validation Testing: In some regulated industries, this term used to be used to indicate the testing that verified that the performed to meet the business’s needs. If you are in a regulated industry, it is no longer a good idea to refer to this testing in this manner, as “validation” is no longer just a set of tests, but a way of controlling an entire process. “Validation tests” are also not the scripts that you get with your software to verify that the software works properly, nor any other type of “canned” scripts that you might purchase. That said, there are the actual tests required for the FDA-regulated industries, and the rest of the industries do a variety of testing to ensure that they basically have the system working to meet their needs. This is sometimes also referred to as “Systems Integration Testing” as it is an opportunity to ensure that all the pieces work together as planned. In any case, this testing requires knowing enough about the process from the beginning to the end to take the data through the system for various cases. So, back to the original question: How much testing is enough? If you are in a regulated industry, you must test enough to satisfy your regulatory agency that you have installed, configured and customized your system to your own expectations. If you are putting your system into a manufacturing area, you must test until you can satisfy yourself that you will not be putting in a system that will negatively affect production time. In any system, you need to test the system well-enough to verify that your users will get more up-time than down-time. You must also provide them a system that helps them in their work more than it hinders them. In some cases, it is a combination of these answers. Even beyond that, each person that writes a test script or that performs a test must be competent to do the job or the actual test is not going to mean much. We must test to verify something, not just for the exercise of adding work to the schedule. The testing, itself, must add value to the process. As a customer of mine pointed out, if you are not comfortable signing your name to a piece of paper that said that you tested something thoroughly, it is probably a sign that you did not test it properly. On the other hand, no-one can sign a document that says that they have produced something that is error-free. As that old adage says: “The last bug isn’t fixed until the last user is dead,” because as long as there are people are using the system, there will always be one more bug out there for them to find. |
||
| © GeoMetrick Enterprises 2001 - 2006 |
|
|