- Principle 1: Definition To test a program is to try to make it fail.
- Principle 2: Tests versus specs Tests are no substitute for specifications.
- Principle 3: Regression testing Any failed execution must yield a test case, to remain a permanent part of the project’s test suite.
- Principle 4: Applying oracles Determining success or failure of tests must be an automatic process.
- Principle 4 (variant): Contracts as oracles Oracles should be part of the program text, as contracts. Determining test success or failure should be an automatic process consisting of monitoring contract satisfaction during execution.
- Principle 5: Manual and automatic test cases An effective testing process must include both manually and automatically produced test cases.
- Principle 6: Empirical assessment of testing strategies Evaluate any testing strategy, however attractive in principle, through objective assessment using explicit criteria in a reproducible testing process.
- Principle 7: Assessment criteria A testing strategy’s most important property is the number of faults it uncovers as a function of time.
Switching context to the subject of testing (computer) systems in a broader sense, I am reminded of a presentation at a workshop that I attended a few months ago (it was more of a penetration testing presentation):
Here is how you learn more things about your system:
- You can have a committee study your system and plan testing; they will get back to you in 30 years.
- You can plan, execute exercises and evaluate results for two years prior to putting it to production.
- Or you can have a real life crisis and learn everything you need in two hours.
That is why you need to launch test events to the largest userbase possible.
- The ultimate (and sometimes only) test is final deployment.
Update: One year after the “Seven Principles of Software Testing”, Meyer blogged “What is the purpose of testing?“