Risks, feelings, management

What if everyone –our managers, auditors, testers — got it wrong with the topic of risks? 

What if what we witness regarding risk management  is just a charade ?

Long ago, I read Jerry Weinberg say, “The definition of quality is always political and emotional.”  That resonated with me then and still does.

I had to get out of a project. At that time I occupied a management position. Usually in taking over a project there are some intermediate steps that need to be done. There was no pressure. It was time for my replacement, let’s call him John, to take it over.  

John was also in a management position. We both worked on the same big suite of products. My project was an integration project, making the communication between products from the suite work. Part of the problem in my project, before taking over, was that it contained hacks. By hacks I mean things which did not belong there at all but on the other products,at the source.Unfortunately, managers from the other products did not want to apply the needed fixes on their parts, to remove those hacks. So all the garbage needed to be fixed on their part was thrown on this integration project, before I came. 

My team fixed these problems in both the integration product and the other products from the suite. 

As I reflect back on my conversations with John,I remember several things that appeared “off.” He seemed  very detached about topics we needed to discuss in order for him to take up the project .  My perception was that John had exaggerated detachment. When he spoke with me everything was ok. But with the others(management from client , our management, teams) his conversation was all about risks. 

I did not mind at all that John was concerned about risks. What I did not understand in those moments was his lack of convergence. John spoke with me about risks but in a calm way. When he spoke with others he was doing it in an alarming and exaggerating way. Also I have noticed certain resistance from the teams he had. Anyway that exaggerated perceived detachment in combination with how he handled the risks puzzled me. 

Feelings, as much as data, play an important role in decision making. 

Then, I found some scientific research about risks, the risks as feelings hypothesis. This hypothesis says that response to risky situations and how we evaluate them are influenced directly by our emotions/affect. The traditional models of evaluating risks tell us about probabilities and consequences but nothing else. But these, actually, are seriously influenced by feelings.(1)

But there is more. You may wonder why us , in IT, should we care about these studies? Because, is about validated scientific knowledge. Things which other disciplines already studied and proved it. We only have to cross the corridor to these disciplines and learn from them. I like  the word praxis to describe this. 

So, I spoke about the psychological part. But there are also sociological and anthropological studies. These studies show that risk perception has roots also in sociological and cultural elements. For example, the sociological component is telling us how we are influenced in dealing with risks by colleagues, friends, family. The anthropological component tells us that people within groups minimize certain risks and exaggerate others as a way to maintain or control a group.(2)

I realized John had feelings, and those feelings might lead him to assess risks differently than I did. Maybe he felt fear because of the history of the project. 

As said above there is also a sociological part which might help in making sense of this situation. I tried to understand from whom he was taking certain technical information. Who was advising him on this part. And I realized it was about some of  his team members who did not want to be involved in the take over. 

The anthropological component helped me to understand the exaggerated escalation to management. It was a way to maneuver the management groups by playing with risks.

Till that moment I only witnessed a shallow view of how risks were used by management. It was an eye opener to me regarding risks and how they should be used in management, testing, audits. 

I mentioned audits. I would like to see an auditor knowing these aspects of risks. Maybe this will help in order to be able to see the spirit of the law and not just the letter of the law. Risks combined with the tacit knowledge might help understand the why’s of why certain risks are articulated, handled and made public in a very specific way. Audits is not about simple receipts ignoring the context and professionalism of a discipline.

Emotions play an important aspect in, risks, judgment/evaluation/assessment. How right Jerry Weinberg was.


(1) Risks as Feelings, Loewenstein, George F. and Weber, Elke U. and Hsee, Christopher K., Psychological Bulletin, Vol. 127, 2001, Available at SSRN: https://ssrn.com/abstract=929947 

(2) Perception of Risk, Paul Slovic, Science 17 Apr 1987:Vol. 236, Issue 4799, pp. 280-285, https://science.sciencemag.org/content/236/4799/280

Tacit knowledge, testing, requirements, management

Have you ever wondered why it does not make sense to ask or expect  for complete written requirements? 

Do you know why trying to write all test cases to capture the testing needed to be done will bring you to a dead end? 

 

Sometime ago I worked on a big project. There were about 50 people involved. These people were divided into various teams. And these teams handled various products from the entire suite.  

One of those products was hard to install. Imagine that simple settings that could be saved in a config file, like xml or json, were saved in windows registries. The installation of the suite in production was a ‘special’ event on its own. I use the word ‘special’ not in a good sense. Each time an install was being done; people were awake in the middle of the night to stay to test it. But this was one problem in a sea of problems. 

One day, the man responsible for the product installation announced his resignation. His resignation worried me. I asked my colleagues in the office. They weren’t worried. They said, “it’s a different situation, but look, the man who is leaving also writes in a wiki. So, it will be ok”. They had the impression that all what is needed will be made explicit. 

For me this did not make any sense. Not all knowledge can be made explicit. We have a lot of implicit knowledge. Harry Collins in his wonderful book named Tacit and Explicit Knowledge explains why. Imagine that knowledge can be represented like an iceberg. What is above the water is the explicit part and beneath the water the tacit part. So:

  • weak or relational tacit knowledge – Is tacit because of the relationships between people that arise in a social life context. For example:

* concealed knowledge – knowledge which is kept secret intentionally;

* ostensive knowledge – knowledge which is hard to be described/comprehended. Because of that we point to a word, object or a practice to describe it;

* logistically demanding knowledge -imagine a person who knows where everything is, but would not be able to list it if asked), 

* mismatched silences – knowledge kept secret without intention;

* unrecognized knowledge – a person is doing certain things in certain ways. This person would not tell another person because it might not know if these are worth to be told for the second person.

  • medium or somatic tacit knowledge – is tacit because it is incorporated in the human body. How we type, how we juggle, how we balance on a bicycle. What is also important here is about the contrast between conscious and unconscious processing, just think of the process of learning/riding a bike.
  • strong or collective tacit knowledge – is about the social aspects/interactions/relations people have and the knowledge that derives from this. These will influence also the social judgment of why certain things were done. The most important thing is that these things cannot be described and learned by explanation; it must be practiced in a social environment.
 

For me this was the decipher key of this type of situation. It was more than clear to me that even with all the goodwill, that man could not have written everything in a wiki and that things would have escaped to him. And this is ok, normal. I felt calm. I don’t know how to describe this calm; maybe it was a calm because it helped me focus on what I knew would come. 

Do you wonder how the story ended? That moment I anticipated arrived, and it was a difficult moment for my colleagues in that project. So difficult that replacement also suffered enormous stress and left.

Connection with test cases:

In the example above I said about that man who had to write about the steps (and nifty details) related to an installation. But this also applies to requirements, test cases. We have the impression that if we will concentrate all our efforts on writing and managing by test cases is the way to go. Actually, it is a waste, a not so good direction because on testing we do much much more. There are so many aspects which can be covered that it is really hard to articulate it. 

With this overemphasis on test cases we ignore the way we are and how we gain knowledge. Testing encompases many things like learning, critical thinking, experiencing and many others. A lot of this is not explicit but tacit. 

For example, I said about critical thinking, this is for me a form of ostensive knowledge. I used 2 words  to point to an entire discipline not so easy to describe and comprehend. 

Learning about the product involves also understanding why certain decisions were made about that product. Being aware that certain knowledge will be kept secret without intention, mismatched silences or not being aware that an information is so important that needs to be shared-unrecognized knowledge.

Experiencing a certain product might not be easy at all. Information might popup only when discussing with persons involved in the process of working/developing with that product(collective tacit knowledge).

Connection with requirements:

Is helpful to have things written down especially for remembering them. In Scrum, for example, we write to remember but we do not write to communicate the Product Backlog Items(PBI’s) – the PO tells them. 

Also this helps in setting more realistic expectations about written requirements. 

Connection with management:

The story I gave was the moment when I understood the importance of theory and practice (praxis) , in this case regarding some aspects of tacit and explicit knowledge. As a manager it helped me to size up an unexpected situation and also be aware of things that might happen.

Discussion about showing all testing scenarios

Context: Have you ever had a conversation about test automation where the manager wanted you to automate “everything”? Or the manager asked you to show him/her all the scenarios?  And how can we explain it in a simple way? Below is a small/simple dialog trying to do this.  

Manager: Is there a place I can see the testing outputs for all applications? What scenarios are tested and which are not?

Tester: We cannot simply show “all the scenarios”. Testing is not a finite set of scenarios but an exploration of the product to discover new information.

Manager: I do not understand. Everywhere you look they do this.

Tester: Indeed. Look. Maybe there is more to say here. I understand your preoccupation. May I try to offer a perspective? Maybe it will help us. That’s what people say they do. The reality is more complex.

Manager: <<the manager makes a gesture that is ready to listen, but the tester perceived some doubts>>

Tester: I like to use an example that is connected with a problem posed by international terrorism. Yes, it sounds strange and curious at the same time, I think. It was a guy, Max Boisot(1), that tried to explain via a simple analogy why they were not able to catch lots of attacks. Imagine that we have 10 points. These points can represent people, requirements, code packages/branches and other things regarding product development.

Look at this image, please do not stress with math formulas, take it as it is:

Manager: So those points can represent different things in a project?

Tester: yes. Actually just imagine the realities of a project like a woven fabric of different things(events, actions, interactions, feedback, dangers, determinations) that are inseparably associated.

Manager: Ok <<said with a long Ooo>>

Tester: Do you see in the drawing that multiple geometric figures can be created via 2 or 3 or 4  dots?

Manager: Yes, different kind of triangles,  polygons

Tester: Yes, Max Boisot named them patterns. The total number of these is 3.5 trillion. In real life some of them matter, some of them not. But in testing we must look for those that matter through all the existing ones. Somehow I froze at your request because it is impossible for me to give you what you said. I would do it but it is hard if not impossible. 

Manager: Hmmm. And what about automation?

Tester: We can automate a thing which is explicit and deterministic in an algorithmic way. We cannot automate sensemaking, critical thinking, studying, modeling, interpretation. All these , among other things, are part of the process of testing. 

Manager: But every company I look at does this. It seems that every company follows the same thread/trend. Why do they consider it that it can be done, that you can automate everything?

Tester: First , I think that there is a lot of marketing out there. Then how do we expect people who were hired or built  a career or had a certification to admit they were not so right? Is not easy.

Manager: You did not finish the idea with automation and the geometric figures

Tester: Yes. There are three important things:

 – even with the best programers we can’t cover and prove those 3.5 trillion patterns

– I say this because there are different types of coverages(2), I know of at least 100 of them. If you’ll ask a programmer she/he will tell you, maybe, about code coverage.

– As people, we are pattern detectors, etc. If you encourage our agency, we can see patterns in the data/product a tool can’t see.

Manager: But I need a way to show that we are on top of the situation and that we can detect the problems.

Tester: There are ways. For example, to manage testing we can use Session Based Test Management(3). To discover problems we have a long range of heuristics and techniques to do it.

Manager: Ok. But we have to discuss these also. Let’s fix another meeting for this.

Tester: Of course


(1) Connecting the dots, Max Boisot,  https://web.archive.org/web/20091123034027/http://www.hlswatch.com/2009/11/12/nidal-hasan-and-the-problem-of-connecting-the-dots/ 

Note: I searched for the original link of the article “Connecting the dots” but is no longer available. 

(2) Software Negligence and Testing Coverage, Cem Kaner, http://www.badsoftware.com/coverage.htm 

(3) Session-Based Test Management, James Bach, Jon Bach , https://www.satisfice.com/download/session-based-test-management

Scrum Master as a complex dynamical attractor

Context

Some years ago I noticed a strange thing when a new Scrum Master was assigned to a team. That team was already formed and just needed a new Scrum Master.  I observed that team before with another Scrum Master. I also observed the new Scrum Master with his former teams. My first reaction was: “How sad. This team will be transformed in selfish team, capable of doing bad things like sabotage. I saw the good part of these people, now I will see the bad part in them “. And indeed, it was like this.

Problem

I was starting to observe certain patterns of influence of people upon groups of other people, with interesting consequences.

At that time I was wondering how I can explain this in a way that it made sense, was coherent, professional and had some bases.

Why this

This was about a Scrum Master,  but I think it applies for Product Owners, managers, ….

This story is particularly important in context of Scrum. I say this because there is an important detail in the Scrum Guide, which is: “…Scrum (n): A framework within which people can address complex adaptive problems…” (1) (emphasis is mine). Here is about complexity theory, it’s deep stuff.

About the word “theory”

We, the IT, have at our disposition an entire body of knowledge from multiple disciplines which can help us a lot. Unfortunately this is, at least in my circles, rather ignored. And we base our ideas, work, actions on some “beliefs” or blindly.

This word “theory” is about science. Some sciences can be very useful in IT: cognitive science, anthropology, complex adaptive systems theory, etc.

We can use practices informed by theory (praxis)(2) in very practical ways in our day to day work. The fact that we know the theory can help us make sound statements/actions about novel and uncertain situations. We can do this because it is about validated knowledge from sciences(2);

In March, this year, I was at the Scrum training held by James Coplien. Everything he said there was not based on beliefs or impressions. He has done his homework regarding the scientific research. He knew how to back up everything of what he said by theory. I hope more Scrum people will follow his example.

This post is about complex adaptive systems theory. At that time, when I was observing the situation described in the Context section, I was looking at it through the lens of complexity theory, more precisely about attractors.

Atractors

Trajectories  that concentrate  on regular/normal patterns define the idea of system’s attractors. What is found in the basin of attraction will have its behaviours constrained/directed/channeled by the attractor, this means increasing the chance of going in one direction rather than in another. The behaviour of the system is constrained by the dynamics of the attractor. (3)

Can the attractors be persons? Of course, parental presence in a children’s party represent an attractor. Just imagine a children’s party with no parent(4). The presence of the parents there will influence what is acceptable and unacceptable behaviours.

Conclusion

● How a Scrum Master is chosen should not be based on frivolous reasons.  I say this because in complex systems, we deal with social phenomena, when developing projects/products, there will always be unintended consequences. Also in complex systems a small cause does not imply a small effect, not at all.

● Strange attractors are and will be present. The fact that the Scrum Master is a good or bad attractor or not attractor at all, will matter;

● I am so tired of seeing Scrum Masters who are chosen just because:

–  is enough that they know how to do reporting;

– the manager choosing the Scrum Master is enticed by the false appearance of words and behaviours of a certain person wanting to be Scrum Master;

– the person is a tester and is cheaper, in outsourcing, to make testers Scrum Masters and not developers;

– that person is very good in platitudes and reproducing the same things which can be put in just 4 A4 pages;

Scrum is not easy to do, is hard and that’s why I saw so many failures with it. I imagine this is because, maybe, lack of knowledge, fear, political games …of course is not a single root cause….

There is a lot more to say, of course. For example, other anomalies can be spotted in the situation described in the Context or in the points listed in how I saw Scrum Masters being chosen. But the intention of this post was to make the connection between 2 things: Scrum Masters and attractors.


(1) “Scrum Guide” , https://www.scrumguides.org/scrum-guide.html

(2) Dave Snowden,  “Of practical wisdom”,  http://cognitive-edge.com/blog/of-practical-wisdom

(3) Alicia Juarrero, “Dynamic in Action”,   https://www.amazon.com/Dynamics-Action-Intentional-Behavior-Complex/dp/0262600471

(4) Dave Snowden, “The landscape of management: Creating the context  for understanding social complexity”, https://www.researchgate.net/publication/228449006_The_landscape_of_management_Creating_the_context_for_understanding_social_complexity

Inattentional blindness and code reviews

I already spoke about inattentional blindness in the context of testing in a previous post: Inattentional blindness and test scripts. But there are implications also for code reviews.

So, “We asked 24 radiologists to perform a familiar lung-nodule detection task. A gorilla, 48 times the size of the average nodule, was inserted in the last case that was presented. Eighty-three percent of the radiologists did not see the gorilla. Eye tracking revealed that the majority of those who missed the gorilla looked directly at its location.”(1)

But how can the effects of inattentional blindness can be reduced?

”A more effective strategy would be for a second radiologist, unfamiliar with the case and the tentative diagnosis, to examine the images and to look for secondary problems that might not have been noticed the first time through.”(2)

So, it does make sense to do a code review because things might have gone unnoticed. But with complex systems the important thing is the granularity. This means that we have to understand who makes the review and see possible implications.

For example, let’s say a novice has written some code and another novice will make a code review. The junior reviewer, possibly, will understand where and why that code was written like this. But is it enough? I would say no. A senior/advanced developer have to review the code because:

– can share experience;

– has deep technical knowledge(oop, architecture, clean code, properly deal with legacy code, TDD, unit tests, algorithms);

– is capable to have more perspectives.

Is this still enough? I would say no. A tester can take a look at the code with the intent to help him/her with the testing.

There are some lessons here:

– a novice can participate to a review because is a good occasion for him/her to learn. It will also offer a good occasion to be aware of the context and give a feedback;

– is not ok when the code review is not made by a senior because slowly slowly things will get in such a bad shape that it will be difficult to repair;

– a senior developer might miss certain details;

– a manager should not feel comfortable just because the process does not allow a commit of code in source control without reviews. Even if that process might say something like this: “a review must be made by at least one senior developer”;

– it is related to testing in the sense that the tester is searching for information which will be used in designing the tests;(3)

– this should not be used by developers as an excuse for a lack of professionalism.

This is praxis(4)….


(1) Trafton Drew, Melissa L.-H. Võ, and Jeremy M. Wolfe, Psychological Science September 2013;24(9):1848-1853, “The Invisible Gorilla Strikes Again:   Sustained Inattentional Blindness   in Expert Observers”: http://search.bwh.harvard.edu/new/pubs/DrewVoWolfe13.pdf

(2) Daniel Simons Christopher Chabris, “The Invisible Gorilla” https://www.amazon.com/gp/aw/d/B009QTTYCQ

(3) James Bach, Rapid Software Testing class discussion [2017]

(4) Dave Snowden,  “Of practical wisdom”:   http://cognitive-edge.com/blog/of-practical-wisdom/

Inattentional blindness and test scripts

“We asked 24 radiologists to perform a familiar lung-nodule detection task. A gorilla, 48 times the size of the average nodule, was inserted in the last case that was presented. Eighty-three percent of the radiologists did not see the gorilla. Eye tracking revealed that the majority of those who missed the gorilla looked directly at its location.”(1)

We think we pay attention to things but it’s not like that. We’re aware of only a small part of what we see at any moment. So we might even look at something and not be able to see it – it seems eyes actually look at something but without actually seeing it.

Initially, when I thought about test scripts in conjunction with the concept of inattentional blindness, I imagined it was a way to disprove test scripts entirely. I say this because of the following logical reasons:

– If a test script is being followed with great precision, then for sure other things are overlooked. Therefore, this means the script is not a very useful technique;

– But if the tester says that it will also check other things that are not included in the test script, in a way this behaviour invalidates the test script.

In other words, these two approaches practically invalidate the idea of a test script.

I admit I was biased with test scripts because I saw it (and I still see it) applied in a very very wrong way.

James Bach surprised me by saying that, actually, test scripts are an advanced test technique that should not be done by novices(2).

An advanced tester is aware of his/her limitations and makes sure that the testing is not affected in a bad way when using test scripts.

The reason why a junior tester should not receive test scripts is because, the likely scenario is that the test scripts will not be run as they should be(2). A junior/novice tester should be guided/accompanied by a senior tester.

This is praxis(3)….


(1) Trafton Drew, Melissa L.-H. Võ, and Jeremy M. Wolfe, Psicholigy Science September 2013;24(9):1848-1853, “The Invisible Gorilla Strikes Again:   Sustained Inattentional Blindness   in Expert Observers”: http://search.bwh.harvard.edu/new/pubs/DrewVoWolfe13.pdf

(2) James Bach, Rapid Software Testing class discussion [2017]

(3) Dave Snowden,  “Of practical wisdom”:   http://cognitive-edge.com/blog/of-practical-wisdom/