Rapid Software Testing

Round Earth Test Strategy and earthquakes

Recently, James Bach published a nice analogy/model regarding test strategy, named “Round Earth Test Strategy“(1)

Early in 2018, in January, James Bach was kind enough to share this analogy on a special channel RST( this channel was created especially for RST alumni; it’s an active and very very very useful group to be in – another very good reason to take RST class(2)).

I liked it. But, in that moment, and days/months after, I felt that I still miss some aha moments, about those things, which confirm in a way that a certain subject is internalized in myself somehow.

Museum

In the image of this blog post is the Natural History Museum in London. It was here where I had the aha moments I knew I was missing. Suddenly, in my head, I remembered James Bach’s round earth model. In there is a section named “Volcanoes and Earthquakes”. When I began to read what was written, I was amazed how the things described there can be used as an analogy, as James thought, for testing, but maybe also for problems/risks/bugs/tools/approaches from the IT field:

● “Earthquakes can happen without warning, causing death and destruction on a massive scale. When they strike we feel a sudden, violent shaking of the ground, but they are caused by slowly moving plates on Earth surface. As these plates move, pressure builds up until it finally gives away”(3) → In reading this text I remembered bugs. How certain bugs can create a lot of mess (urgent calls, staying late in the night, missing important moments). How those bugs appear suddenly, without notice. But what stroke me is the fact of slowness and violence of earthquakes.

Slowness in:

– how functionality is written → It’s not like the functionality is developed instantly. There are meetings, mails, writing code, searching through code, testing, etc.

– execution of the program – managed memory leak, usually, takes time to be observed. But when it reaches a critical point, entire system crashes not only the programs.

The idea of moving plates made me think of integration testing.

● “Preparing for the worst; living with earthquakes: Scientists can’t predict exactly where and when an earthquake will strike but they know roughly which area around the world are at risk. It is vital for the people living in these areas to prepare for what may come and know what to do when it does. Without adequate preparation, earthquakes can cause huge suffering and destruction”(3) → It describes perfectly the role of testers and why they are searching for the worst, why they should think negatively. Testers, like these scientists, should understand that they cannot predict and also can’t be sure that the software is ok (just a black swan among thousands and thousands of white swans was enough to proof that there were not just white swans). Although they can’t predict or prove the correctness of a software program, they will use models to identify possible areas with problems guided by risks (For example, by looking at the source control metadata, weak areas within the source code can be discovered. ). Since we’re talking about people, the risks also have a psychological and sociological dimension. It is sure, problems will occur and maybe we should guide our testing also by the possible suffering we create, for the ones using our software.

● “Impact scale: There are different ways of measuring earthquakes. Unlike Richter scale, which measures magnitude of the shaking, the Mercalli scale measures the amount of damage caused – the loss of life and the damage of buildings. Generally speaking, the higher earthquake magnitude the greater the devastation, especially when it strikes near populated areas. But you also have to factor the in depth of the earthquake, and how well people have prepared. A big earthquake can have a low Mercalli value if it happens deep underground or if buildings have been properly supported”(3) → When I saw measuring, I recalled the nonsense in counting the test cases – which is very susceptible to the reification error and this is very, very dangerous. But we have something which tries to avoid the reification error and is based on events/activities: it is called Session Based Test Management.

But there is more, the fact that a big earthquake can have a low Mercalli, made me think about complexity and the fact that the relation between cause and effect is not linear. Populated areas also indicates complexity, because social systems are inherently complex. This means, for testing, that the approach is more informal (it’s about trying/probing, then make sense of it, then respond/report the possible problems which might occur), not a formal one – when thinking of testing, and more specifically the checking dimension, maybe here mutation checking makes sense.

There is also another implicit dimension here, which is the place where the earthquake happened. Even if it is a small earthquake but it happens in the ocean, it can generate a tsunami. If I relate this to testing, it makes me think of the different coverage areas like structure, platform, function, operations, data, time, interfaces(9), hazard.(thx Ionut Albescu)

● “Danger after the quake: The danger doesn’t stop once the ground has stop shaking. Fires, landslides and even liquefaction can all cause damage and loss of life…Scientists and engineers have developed ways to deal with these dangers through defenses, warning systems and building design. But even with the best plans in place some communities can still be caught off guard”(3) → How many developers, testers, scrum masters,… think that maybe a person will be fired because he/she is not working fast enough with our software product? How will a developer sleep at night when his/her code caused, even indirectly a death, or a bankruptcy? There are consequences, but a lot of people don’t get the fact that they must assume also unintended consequences, which were triggered by what their product does. How will they deal with that?

They have developed defences, but it’s interesting that they speak about them with terms like: tools (4), models(5). We, in IT, use a lot the word “automation”…

When they speak about plans it’s very serious because it’s about people’s lives. They are not using some tools/techniques as a plan, they are guided by the reality of the situation. What a test plan means, for a lot of people: automation at the unit level, integration and maybe acceptance(BDD). And they add exploratory, although they are not able to articulate what it means and how to show/do it in a professional way → this is not a test plan.

What if aftershock are the equivalent of hotfixes? And we have a flow like this:

1. A bug appears

2.A quick hotfix is made, but in a hurry

3. As a result, that hotfix can cause undesired problems(maybe inadvertently), because of the chaos created by the initial bug.

The last sentence made me think at the japanese word “hansei”. Because even though those scientists built/are building, defense and warning mechanisms(kaizen), the things can still go wrong and this is sadness/regret.(6)

● “After the earthquake, responding to disaster: earthquakes and tsunamis can destroy home and buildings, transforming lives. The hours and days that follow the disastrous events can be vital for saving anyone who has been trapped as a result….As people come to terms with the destruction they can start with the process of building resilience – changing the way they live and act to deal with the risk of an earthquake in the future. This can leave them better prepared for future earthquakes“(3) → The keyword here is resilience, but a lot of IT people want robustness. We, in IT, have chosen the wrong metaphor. Rapid Software Testing(a Context Driven methodology) is fully aware of that, that’s why it’s so different from “Factory Style” testing(7), because it sees the context as an ecology, not as a factory(8).

Conclusion: Read James Bach’s post, then read the text from the museum again. I hope you will find it as useful as it was for me.

(1) James Bach, “Round Earth Test Strategy”, http://www.satisfice.com/blog/archives/4947

(2) https://rapid-software-testing.com/

(4) “Tectonic hazards/Earthquake engineering“, https://en.m.wikiversity.org/wiki/Tectonic_hazards/Earthquake_engineering

(5) “Improving defence against earthquakes and tsunamis”, https://www.ucl.ac.uk/news/2017/mar/improving-defence-against-earthquakes-and-tsunamis

(6) James Coplien, Interview, www.infoq.com – in this interview he explained scrum and these 2 japanese words, among other things – I can’t find the text, as link, but I have it as text, tough, in my personal archive.

(7) James Bach, Michael Bolton, “RST Appendices”, http://www.satisfice.com/rst-appendices.pdf – pages 3-6

(8) Alicia Juarrero, “Safe-Fail, NOT Fail-Safe”, https://vimeo.com/95646156

(9) James Bach, Michael Bolton, SFDIPOT, http://www.satisfice.com/rst.pdf

Why would you automate, in testing, everything?

A bit of context: I was trying to answer a question Michael Bolton put on the Rapid Software Testing Slack chat group, the question was “Why would you automate anything?”. Yes, it was the word “anything” not the word “everything”. In trying to respond to the original question I answered also to the question from the title of this post. In my head was a mixture of those two different words, strange…

Below are several reasons I spot it about why is this desire to automate, well to automate everything :

– some managers does not have a clue what professional testing is ( by professional testing I mean Rapid Software Testing a Context Driven Methodology). The managers still think of it in manufacturing terms(” automate as much as we can”). And because of this then they want a way to speed up things. They see testing as tangible stuff. When I say management, I’m not referring only at the first level, but at all levels in an IT company;

– some testers who are not aware of their profession. Is a trend now with (using) tools, who encourage this. It seems the marketing promoting the tools is doing a good job. And novices (we should not confuse years worked with years of experience) think this is the way. Then with the help of uninformed management we arrive at the desire to automate everything. Somehow these testers are not aware of the misinformation they are doing, is not an intentional thing;

– some testers who see the trend( the dominant thinking) and for their personal advantage go on this path. They don’t care about the interest of the project. This is sick and a unprofessional behaviour which I witnessed several times;

– some programmers who are ignorant of what a tester really does. They think, because of their arrogance, that are too smart to need a tester or to respect him/her. For me this enters in the unprofessionalism behaviour;

– some programmers with good intentions seeing the benefit in automation but trying to make a general rule without having other thoughts. For sure these are not senior/advanced developers. I say this because a real senior/advanced developer knows the plethora of the possible problems and they shut up and think, holistically, before speaking/making a decision;

– a teacher/mentor/manager making their people to think about what it means to automate everything. I can extend here the idea that a tester/programer might do this do investigate/search/discover more about this topic.

What a Scrum Master should or should not do

In my circles, I see lots of confusions/discussions regarding what a SM should or should not do. I already wrote before about the need of a SM for a project. Sometimes I feel this role is trivialized, that is being used as excuses of not doing things.

“Most arguments seem to be about conclusions, when they’re really about premises”(1) . So the premises:

● Beside the role as a SM or PO, it might happen that the organization will consider the SM or PO also in a position of middle management (PM, TL). But the discourse regarding PM/TL is changed a little bit: The PM/TL has to take care the project is OK, whatever that would mean…

This might contradict the Scrum thinking/philosophy, but this is another story. The idea is that we have to deal with this reality and acknowledge it.

● The Scrum Guide describes SM relation with PO, team and the organization. The Scrum guide might seem simple, but actually I feel it as a rather exceptional condensation of a lot of information. Take for example just the word “facilitation”, I think days can be spent only on treating and understanding this word.

I have some, let’s say, rhetorical questions:

– Shouldn’t a, good, really good, not mediocre, SM understand PO work in order to help him? By this I mean, maybe, to acquire skills of PO. The better the SM understand the PO, the better SM he becomes because he’ll be able to be more helpful for PO (and subsequently for the team);

– Should the SM help the team by removing impediments and creating high value products only by ‘organization means’ and by encouraging them(being supportive)? Where is his personal touch? By ‘organizational means’ I mean that the organization should help the SM no matter the problem, of course via standard tools/ways of working to solve the problems. SM should be creative, be autonomous and adapt solutions to the specificity of each project.

How the SM will really coach the team in self organizing and cross functionality if it does not participate in the current work? Should encouraging with platitudes be enough? Bringing others to do it? Hmmm….Maybe encouraging without facts and actions will have a small impact.

What I want to say is that, maybe maybe, we need to consider that a SM should be also technical. By technical I mean a developer or a tester(I belong to the Rapid Software Testing testing school just to be clear what I mean by tester) or …, so she/he can take info not only from others(like horoscope) and numbers(we are mature people still)(4). I think I would phrase it more generally that they should be with the “skin in the game”.

Yes is possible to have the non-technical SM…but…but…this will be helpful, for SM, as rather than rely entirely on the input of the others, SM will be able to make his/her own assessment based on his/her experience and validate it and not take it for granted.

I saw that in delicate situations where a project must be saved a good multi-skilled SM is being put into play, when the others kind of SM are being let aside maybe because is safer…

I feel there is a tendency to put as SM people with no or little experience/knowledge but with good discourse…. – that’s why I am tough! I try to imagine a surgery team in a hospital organized by Scrum, then the SM for that surgery can be the physiotherapist? Just to be clear: Scrum is good, but in the proper boundary.

● Tuckman model(*) of team formation and development. The situational leadership model is changing depending on the stage the team currently is in. So it might be directing or coaching or supporting or delegating. So within the same Scrum team the tactics of SM might or should change.

● Last, but not least, I think is imperiously needed to be aware of the “multi-ontology sense making”(2). I like that this invites us to transcend the type of system we are in. Scrum, in Cynefin(3) parlance, is at the boundary between complex and complicated. But a dynamic of project/situation will not always stay in a certain type of system. What if suddenly from complicated the type of the situation will change between chaos and complex. What should a SM do? Stay and use the techniques/tactics for which Scrum is designed? Shouldn’t try to adapt and consider the appropriate techniques/tactics/measures of action for the corresponding type of system? Ok, it will no longer be SM by the book, so?

So, I hope to see more SM’s:

○ who are very well prepared;

○ that stop using platitudes;

○ that know about Nonaka work ( even if they my not agree with some of his work );

○ who know the work of Jim Coplien (dailys are his idea/inspiration from Scrum; and the technical book he wrote “Lean Architecture: for Agile Software Development” is great);

○ who stop trying to be pseudo-psychologists;

○ who understand that to be a coach/mentor requires preparation, lot of it;

○ who understand that agile does not equal Scrum;

○ who dare to ‘dirty’ her/his hands and to help the team by working effectively in the project, of course, making use of the skills they have when she/he thinks that the benefits of this action are bigger than the possible disruptions;

○ who will have no problem to let another one be SM and she/he be a simple member of the development team, at least temporary;

○ who will realize that she/he is Master not after a few days of training, but after years and years of work and preparation;

○ who understand the job of a developer and the job of a tester, and then realize that you cannot switch one with another with the excuse of self-organization;

I know there are other schools of thought and is ok. I accept their opinion, but this does not imply that I also agree with them. Context matters a lot.

(*)Update February 27, 2018:

I indicated Tuckman model because in the agile world is a very known model. My intention was to show that a team is not in a single state at all, but on the contrary.

I reviewed the original paper where Tuckman wrote about this. Actually this model is not proven. In the initial paper “Developmental Sequence in Small groups”(5) he reviewed the existing literature regarding “dealing with developmental sequence in small groups” and he suggested “fruitful areas for further research”. So is a “statement suggested by data presented and subject to further test”.

Ten years after the original paper was published another one was published, “Stages of Small-Group Development Revisited”(6), in order “to examine published research on small-group development done in the last ten years that would constitute an empirical test of Tuckman’s hypothesis”. In the conclusion is stated that “It is noteworthy that since 1965 there have been few studies that report empirical data concerning these stages of group development…There is a need to supply statistical evidence as to the usefulness and applicability of the various models suggested in the literature”

(1) Michael Bolton, “Testing, Checking, and Changing the Language”: http://www.developsense.com/blog/2009/09/testing-checking-and-changing-language/

(2) Dave Snowden, “Multi-ontology sense making; a new simplicity in decision making“ http://cognitive-edge.com/articles/multi-ontology-sense-making-a-new-simplicity-in-decision-making/

(3) Dave Snowden, Mary E. Boone, “A Leader’s Framework for Decision Making” https://hbr.org/2007/11/a-leaders-framework-for-decision-making

(4) James Bach, “No KPIs: Use Discussion”, see comments, http://www.satisfice.com/blog/archives/1440

(5) Bruce W. Tuckman, “Developmental Sequence in Small Groups”, http://dennislearningcenter.osu.edu/files/2014/08/GROUP-DEV-ARTICLE.pdf

(6) Bruce W. Tuckman Mary Ann Jensen, “Stages of Small-Group Development Revisted”, http://openvce.net/sites/default/files/Tuckman1965DevelopmentalSequence.pdf

Incapable to defend real testing

I am a developer, also a Team Lead. I remember now the moment when I realized that if I’m a better developer is also because of the help of a good testers. Initially it was like a cognitive dissonance: how come a good tester help me be a better developer? Even stranger for me was that when I realised this, I was deep involved (still deep involved) with TDD, unit tests and Automated Acceptance Testing.

There are devs who think that if they do tdd/unit tests everything will be ok and testers will do “some clicks”. Or even worse, the testers, for those devs, should check all the nonsense made by a developer, because that developer will not test. So strange. So a developer should not experiment, learn, explore, investigate, find relevant information from his/her own work.

I am sad. I was unable today to defend the real craft of testing (Rapid Software Testing). In a way, I felt that I was not able to defend those testers who contributed to make me better professional. So strange feeling.

Then I begun to think more. Now is a lot of talk regarding “manual” testing and “automated” testing.

Note: I told to a marketing person that she is doing “manual” marketing. And this sound so strange almost like offending. I told her about testing and “manual” testing and she understood. Still I can’t forget that it almost sound offending.

Why do they say “manual” testing? Maybe because a tester should just verify a specification, no matter the form of that specification – wait, I am wrong, is a user story with acceptance criteria, of course. Since is a clear specification, it is imagined, I think, that they can also write what they will test – some will say they write the test…. Hmmm…So, if they can write it down this means they don’t need a tester with experience – may be temporary – , a junior should be enough because anyway acceptance criteria are done by the Product Owner. But if a junior can do it, then … wait we can automate those steps, yes, yes via UI. So, hmm, why not get rid also of the junior tester, because we have the automation which is the ultimate goal actually. A click of a button and that’s it. For sure that automation code will be developed not by experienced developers like it would be the production code, no, no we have “automation” testers.

So why a tester should matter so much when actually he/she is only doing some of the shallow testing a developer must do? Shallow and clear steps which can be done by anyone.

So, I imagine that’s why – briefly described – some testers are being called manual and automated.

If testers spend time on shallow bugs they will not have time to see for deeper ones. Such deep that neither a dev can’t see it.

But what do those testers that I was not capable to defend? They, although they might not phrase it like this: investigate, doubts, searching for information, pose questions, questions everything, do more than confirm acceptance criteria, they challenge the actual criteria, they make definition of done/ready relative when all the other thinks are clear and absolute, are able to begin testing without specifications if needed, be able to show the multidimensionality of a problem/specification/story, use any tools which can help them (not just selenium or…), they will not answer to different problems with same answer because they are aware of context, support developers, …. please see here for a much longer list: http://www.developsense.com/blog/2017/04/deeper-testing-2-automating-the-testing/

Sorry colleagues testers :(.