Seven Principles of Software Testing

Bertrand Meyer (of Eiffel fame) writes about the “Seven Principles of Software Testing“:

  • Principle 1: Definition To test a program is to try to make it fail.
  • Principle 2: Tests versus specs Tests are no substitute for specifications.
  • Principle 3: Regression testing Any failed execution must yield a test case, to remain a permanent part of the project’s test suite.
  • Principle 4: Applying oracles Determining success or failure of tests must be an automatic process.
  • Principle 4 (variant): Contracts as oracles Oracles should be part of the program text, as contracts. Determining test success or failure should be an automatic process consisting of monitoring contract satisfaction during execution.
  • Principle 5: Manual and automatic test cases An effective testing process must include both manually and automatically produced test cases.
  • Principle 6: Empirical assessment of testing strategies Evaluate any testing strategy, however attractive in principle, through objective assessment using explicit criteria in a reproducible testing process.
  • Principle 7: Assessment criteria A testing strategy’s most important property is the number of faults it uncovers as a function of time.

Switching context to the subject of testing (computer) systems in a broader sense, I am reminded of a presentation at a workshop that I attended a few months ago (it was more of a penetration testing presentation):

Here is how you learn more things about your system:

  • You can have a committee study your system and plan testing; they will get back to you in 30 years.
  • You can plan, execute exercises and evaluate results for two years prior to putting it to production.
  • Or you can have a real life crisis and learn everything you need in two hours.

That is why you need to launch test events to the largest userbase possible.

Oh well, as Peter Viscarola pontificated from the NT Insider, people “just write some code“. Which leads to:

  • The ultimate (and sometimes only) test is final deployment.


Update: One year after the “Seven Principles of Software Testing”, Meyer blogged “What is the purpose of testing?

2 thoughts on “Seven Principles of Software Testing

  1. As many zero budget developers know, when one goes without proper testing labs or QA people then the seven stages get compressed into two stages:

    A) Write the code , show it off , catch some bugs.
    B) Deploy because you are behind schedule and fix on the fly.

    One is always behind schedule, the service ought to have been out the door two years ago , no matter if the developer was asked or not.

  2. Well Meyer is the basic proponet for Design by Contract. Zero budget developers coding in Eiffel could avoid such stuff. I am the first to admit that doing so is not an easy task (I was exposed to Eiffel (on Ultrix!) back in 1993 with no success despite YV‘s efforts).

    However your comment above did not only describe zero budget developers and small software houses, but a significant fraction of the Greek software industry who by the way consider themselves big and serious players :)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s