Category: Testing

Inattentional blindness and test scripts

“We asked 24 radiologists to perform a familiar lung-nodule detection task. A gorilla, 48 times the size of the average nodule, was inserted in the last case that was presented. Eighty-three percent of the radiologists did not see the gorilla. Eye tracking revealed that the majority of those who missed the gorilla looked directly at its location.”(1)

We think we pay attention to things but it’s not like that. We’re aware of only a small part of what we see at any moment. So we might even look at something and not be able to see it – it seems eyes actually look at something but without actually seeing it.

Initially, when I thought about test scripts in conjunction with the concept of inattentional blindness, I imagined it was a way to disprove test scripts entirely. I say this because of the following logical reasons:

– If a test script is being followed with great precision, then for sure other things are overlooked. Therefore, this means the script is not a very useful technique;

– But if the tester says that it will also check other things that are not included in the test script, in a way this behaviour invalidates the test script.

In other words, these two approaches practically invalidate the idea of a test script.

I admit I was biased with test scripts because I saw it (and I still see it) applied in a very very wrong way.

James Bach surprised me by saying that, actually, test scripts are an advanced test technique that should not be done by novices(2).

An advanced tester is aware of his/her limitations and makes sure that the testing is not affected in a bad way when using test scripts.

The reason why a junior tester should not receive test scripts is because, the likely scenario is that the test scripts will not be run as they should be(2). A junior/novice tester should be guided/accompanied by a senior tester.

This is praxis(3)….

(1) Trafton Drew, Melissa L.-H. Võ, and Jeremy M. Wolfe, Psicholigy Science September 2013;24(9):1848-1853, “The Invisible Gorilla Strikes Again:   Sustained Inattentional Blindness   in Expert Observers”: http://search.bwh.harvard.edu/new/pubs/DrewVoWolfe13.pdf

(2) James Bach, Rapid Software Testing class discussion [2017]

(3) Dave Snowden,  “Of practical wisdom”:   http://cognitive-edge.com/blog/of-practical-wisdom/

Is not ok to define a tester as a manual tester

Let’s try an experiment(1):

Scrum Master = Manual Scrum Master

Programmer = Manual Programmer

Project Manager = Manual Project Manager

Product Owner = Manual Product Owner

Team Lead = Manual Team Lead

Technical Lead = Manual Technical Lead

Delivery Manager = Manual Delivery Manager

Security = Manual Security

Marketing = Manual Marketing

Coach = Manual Coach

Trainer = Manual Trainer

Mentor = Manual Mentor

CTO = Manual CTO

CEO = Manual CEO

Is not ok, isn’t it?

Maybe this is happening because it is not known what a tester should really do and what testing, professional testing, actually is.

But is also sad that with all the agility we left the professional testing outside and this is not ok.

I am glad that Robert C.Martin has spoken about this. In a way he begun to heal the divide between agile and professional testing (Context Driven Testing, Rapid Software Testing). Below you can find reproduced an entire blog post written by Robert C. Martin:

“James Bach gave a stirring keynote today at ACCU 2010. He described a vision of testing that our industry sorely needs. Towit: Testing requires sapience.

Testing, according to Bach, is not about assuring conformance to requirements; rather it is about understanding the requirements. Even that’s not quite right. It is not sufficient to simply understand and verify the requirements. A good tester uses the behavior of the system and the descriptions in the requirements, (and face-to-face interaction with the authors of both) to understand the motivation behind the system. Ultimately it is the tester’s job to divine the system that the customerimagined; and then to illuminate those parts of the system that are not consistent with that imagination.

It seems to me that James is attempting to define “professionalism” as it applies to testing. A professional tester does not blindly follow a test plan. A professional tester does not simply write test plans that reflect the stated requirements. Rather a professional tester takes responsibility for interpreting the requirements with intelligence. He tests, not only the system, but also (and more importantly) the assumptions of the programmers, and specifiers.

I like this view. I like it a lot. I like the fact that testers are seeking professionalism in the same way that developer are. I like the fact that testing is becoming a craft, and that people like James are passionate about that craft. There may yet be hope for our industry!

There has been a long standing frission between James’ view of testing and the Agile emphasis on TDD and automated tests. Agilists have been very focussed on creating suites of automated tests, and exposing the insanity (and inanity) of huge manual testing suites. This focus can be (and has been) misinterpreted as an anti-tester bias.

It seems to me that professional testers are completely compatible with agile development. No, that’s wrong. I think professional testers are utterly essential to agile development. I don’t want testers who rotely execute brain-dead manual test plans. I want testers using their brains! I want testers to be partners in the effort to create world-class, high-quality software. As a professional developer I want – I need – professional testers helping me find my blind spots, illuminating the naivete of my assumptions, and partnering with me to satisfy the needs of our customers.”(2)

(1) Michael Bolton, “Manual and automated testing”: http://www.developsense.com/blog/2013/02/manual-and-automated-testing/

(2) Robert C. Martin, “Sapient Testing: The “Professionalism” meme”: https://sites.google.com/site/unclebobconsultingllc/home/articles/sapient-testing-the-professionalism-meme

A fallacy when estimating in Sprint Planning

Usually, in sprint planning comes a moment when developers and testers have to give an estimation. After the tasks seem to be clear, the testers and developers give their corresponding story points. What seems strange to me was the fact that these estimations are calculated and a single number is created.

For me, this approach seems like comparing apples with pears and then generating a number.

I imagine a lot of agilists will say that this is working. I would say: maybe. What gave them the impression it works? Well, I saw that human nature is capable to cope with formal non-sense, so that, then, informally, make it work, one way or another. Why? Well, because, sometimes, a lot of energy is lost in convincing some people of some strange truths. Truths that are not in trend but for the moment might seem easier to cope with. Why? Well, because of laziness, fear, workload, or just because it’s safer….

For me, developers and testers worlds are very different. Yes, these worlds can intersect somehow, but they are still very different like two oceans uniting/meeting each other.

For a developer, when he/she knows what to solve and how to solve it, the task is easy, and the code is written. That code should behave in a deterministic manner. The developer might write unit tests and/or integration tests – from a tester perspective these are automated checks not tests(1). And when I say developer I am thinking of an XP developer, of a developer who should aim at what Uncle Bob says(2).

For a tester things are different. Although things might appear to be clear for the same story, we might deal with non-determinism. When I say tester I am thinking of a tester who is part( consciously or unconsciously) of Context Driven Testing(3) school, for example a Rapid Software tester(4). Testing actually poses a need for a very different skillset because the problems the tester encounters have a different nature, compared to the ones a developer faces. So even if a developer has modified just one line of code, a tester can spend days and days if she/he decides to make a deeper testing, and not just a shallow one. Why? Because the code is, usually, surrounded by a larger code context that might affect the stability of the product. But if is just one line of code, why would it take a lot of time? A small cause can have big effects. For example, does the developer know with great precision all the details, internal technical details of the code? A simple data type problem can cause one of the strangest and ugliest effects and it all starts from just one line of code.

Therefore, I think we should be careful what we do, and we shouldn’t be reluctant to raising questions, even if we might be wrong. I consider this approach of combining numbers that mean different things, a fallacy. For me, this is an oracle(5) to spot the problem and, why not,  I have a name for it: Two oceans integration.

(1) James Bach, “Testing and Checking Refined“: http://www.satisfice.com/blog/archives/856

(2) Robert C. Martin, “The Programmer’s Oath”: http://blog.cleancoder.com/uncle-bob/2015/11/18/TheProgrammersOath.html

(3) Context Driven Testing: http://context-driven-testing.com

(4) Michael Bolton, “What Is A Tester?”: http://www.developsense.com/blog/2015/06/what-is-a-tester/

(5) Michael Bolton, “Oracles from the Inside Out, Part 1: Introduction”: http://www.developsense.com/blog/2015/09/oracles-from-the-inside-out/

A programmer learning testing

I begun to learn about testing because I was blessed to work with some really good testers in my life. Because of that, I wanted to learn more, in order to make my interaction with them more fruitful. If I am a better developer it’s also because of them. I am glad that I have met them first, because afterwards, life showed me also the unprofessional testers. This made it possible for me to be able to compare, analyze, see…

Learning about testing helped me:

  • see through the eyes of the tester;
  • be aware of two worlds that interact with each other, but that shouldn’t be confused;
  • have a better understanding of the professional behaviour that a programmer should have. A programer should test his/her work to the best of their abilities. This way, the tester will have more time to concentrate on rather delicate aspects;
  • respect the tester and his/her job. Sometimes I feel the tester is used to filter the garbage thrown in by a developer doing a lousy job;
  • learn about critical thinking;
  • learn about tacit and explicit knowledge;
  • learn how to approach in a better way the process of creating/building automation checks ( unit tests, integration tests);
  • enter the world of builds and deploys, and aided me to work a lot with them;
  • be careful at semantics;
  • learn about heuristics, oracles;
  • be aware of the work made by James Bach, Michael Bolton, Jerry Weinberg.