I consider the Testing Pyramid a not so good way to use it for sizing up testing situations.
I say this because:
- for each project the response of what can be a test strategy is the same: unit tests, component tests, integration tests and manual tests. But the problems, bugs, issues are contextual. It is like a medical doctor giving the same medicine to each patient, no matter the context.
- automation can be used only for automatic verification of whether something is true or false. But automation in testing is much, much more.
- it invites to a confusion of roles, which means a confusion of intentions. I speak here about the roles of the programmer and tester. The programmer wants to confirm and the tester tries to refute.
- it constrains in an entrenched way regarding the dimensions to consider when testing the product. Important dimensions are ignored, like: product factors, quality criterias, multitude of coverages, product environments – and each of these dimensions have their own internal dimensions.
- it does not invite explicitly to think about the risks that will guide the testing.
- by having unit, integration and UI as the main components of that triangle/pyramid it is like considering that everything is known and that treating the known is enough.
- it invites a false dichotomy of what a human can do and what automation can do, when actually it is a continuum of activities done by the human with the tools.
I kindly ask James Bach what else he might add to the list above. Here is what he added:
- It treats GUI level testing as almost unnecessary, yet real users only experience the product through the GUI. It is wildly optimistic to presume that there are no important bugs to be found in fully integrated products.
- It provides no actual guidance. What does it MEAN to have “a lot of unit level testing” compared to higher level testing? Counting tests is nonsensical. So how is this supposed to be measured? What are we actually supposed to do? Any specific guidance is certainly not based on any kind of science.
- Sometimes it’s called the testing pyramid and sometimes the test automation pyramid. Which is it?
*Regarding the testing pyramid image I just used one of the simplest forms found out there. The testing pyramid concept was created by Mike Cohn
**James Bach proposed and alternative to the Test Pyramid way of thinking called Round Earth Test Strategy . I also wrote a post about it some time ago called Round Earth Test Strategy and earthquakes